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PREFACE 


The kernel for holding this workshop originated at a staff meeting of the 
OTA last year. We were discussing our own involvement in technology assessment 
and strategic planning and how the uncertainty of the future could impact 
judgment. A suggestion evolved for some kind of training experience that 
would enable all who do strategic planning to feel more confident in dealing 
with evaluations of the future. Through some outside contacts, we were aware 
that systematic procedures had been developed and tested that dealt with 
forecasting. These approaches seemed appealing because they involved data 
acquisition, statistics and other mathematical processes which engineers and 
statisticians use. 

Consequently, it was proposed that we investigate the possibility of holding a 
seminar or workshop in 1986 which would expose some of the Lewis staff to 
several of these forecast methods. The investigation revealed that such a 
workshop was feasible and the expertise for leading it was readily available. 
Following the usual NASA contract procedures, a Statement of Work and a 
Request for Proposal were issued. "The Futures Group" were awarded the 
contract. 

What do we expect to derive from this two-day experience? 

1. Each of us can expect to be more aware of several of the methods that 
are used by consultant organizations in forecasting tasks for business 
and industry. We will have some opportunity to execute a few of these 
methods, but our time of experience will be limited; we won't become 
experts. Enough information will be transferred to enable us to try 
to practice on the job what we have learned. 
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2. We should be better able to deal with the future. Even if we don't 
ever execute the methods as they will be taught, we will remember some 
of the parameters that influence forecasting. I expect many of us 

will think differently, at least readjust our thought process, when we * 

are required to make long-range plans. 

0 

3. The workshop has been intentionally restricted to a comparatively 
small number. Nevertheless, this group represents all of the 
Directorates and includes management and non-management staff. 

Collectively, we are a cross section of the Center. It is an unusual 
opportunity for dialogue and interaction. Designed around the 
executing topic of the future, the interaction should be provocative 
and stimulating. At the conclusion of the Workshop, you will have the 
opportunity of judging whether this has been the case. 

The future incorporates all of the major uncertainties we face. 

Primarily, we are not here to learn how to predict it. We are here to learn 
some approaches on how to cope with it, which perhaps may go as far as helping 
to determine it. Research, which is the lifeblood of this Center, is directed 
to the future. The future is the focus of this workshop. By deduction, we 
will be dealing with a topic of vital importance to the Center. 

Those of us who were involved in planning and preparing for this workshop 
have anticipated this day with great expectation. I wish to thank George 
Prok, Loretta Shaw, Hugh Grimes and Betsy Torres from the Office of Technology 
Assessment; John Gregory from the Program Coordination Office; and Dick t 

Clapper, Cindy Forman, and Debbie Griest from the Training Office. 
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Your interest as participants and willingness to commit two days to this 
workshop have inspired and challenged us. The mutual commitments of us all 
cannot help out result in a meaningful experience. 



Robert W. Graham 

Chief, Office of Technology Assessment 
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FINAL REPORT ON FUTURING WORKSHOP 
HELD AT NASA LEWIS RESEARCH CENTER 

October 21 and 22, 1986 


Introduction 

On October 21 and 22, The Futures Group of Glastonbury, CT, ran a "futuring 
workshop" on the premises of NASA's Lewis Research Center under sponsorship of 
the Office of Technology Assessment (Contract No. C-21030). The goal of the 
workshop was to introduce to a varied audience of Lewis personnel, the range of 
forecasting techniques and methodologies available. From the perspective of The 
Futures Group personnel who delivered the workshop, it was a success. The 
audience/participants seemed interested in the topic from the outset and exhibited 
growing enthusiasm as the workshop progressed. The questions and discussions 
during both the plenary sessions and working groups were cogent and challenging 
and reflected a serious interest on the part of Lewis Research Center personnel. 
Perhaps more to the point, the workshop was deemed successful by Lewis personnel 
as well. As is examined more fully on pages 9-11, the evaluations completed at the 
end of the second day rated the workshop as very useful and effective. A very 
large majority indicated that the workshop should be repeated for other Lewis 
Research Center staff. 

Workshop Goals 

The Futures Group designed and presented a Futuring Workshop to NASA's 
Lewis Research Center with four principal objectives: (1) to acquaint the 

participants with the general history of technology forecasting— its generic success 
and failures; (2) to familiarize the participants with a range of forecasting 
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methodologies; (3) to acquaint the participants with the range of applicability, 
strengths, and limitations of each method; and (4) to offer the participants some 
hands-on experience by working through several forecasting case studies employing 
both judgmental and quantitative methods. The goal was not to produce polished 
forecasting experts— that would have been impossible in the time available— nor 
was it to advocate particular approaches. We believed the Workshop would be 
judged a success if the participants gained sufficient insight into technology 
forecasting techniques to (1) understand the range of methods available; (2) make 
judgments about the potential usefulness of various forecasting methods; and (3) 
have an understanding of where more detailed information can be obtained for 
application of the methods to "real life" problems of analysis and forecasting at the 
Lewis Research Center. 

To achieve these goals required, first, that a good deal of information be 
conveyed in a short amount of time; and second, that the format evoke the interest 
of the participants. With respect to Lewis Research Center, our approach involved 
four elements. First, our examples were relevant to many LeRC interests since 
the workshop design and presentation team had backgrounds either in aerospace 
engineering or aerospace-related technology forecasting. Second, wherever pos- 
sible, examples of each method were drawn from previous Futures Group studies 
that are close to interests of the participants. Third, meaningful hands-on 
experience in forecasting was achieved by employing relevant Futures Group data 
bases in quantitative forecast exercises of revenue passenger miles and turbine 
engine technology. Fourth, as part of the workshop materials package, The Futures 
Group provided an extensive technological forecasting reference list including 
available forecasting software so that participants can pursue use of these 
techniques after the workshop (see Appendix A). Finally, copies of all the material 
used in the presentations were made available to Lewis Research Center for 
distribution to the participants. 
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Agenda 


The workshop agenda was designed to accomplish the above goals in a two- 
day conference conducted during regular business hours. The original agenda 
proposed to Lewis Research Center was modified slightly to fit in an extra Delphi 
questionnaire round and to reduce the size of the CONSENSORTM session to a 
more manageable number of people. The schedule below is the schedule that was 
followed during the workshop. 


AGENDA 

NASA Lewis Research Center 
Futuring Workshop 
October 21 and 22, 1986 


Day 1 



8:15 

8:30 

8:45 

9:00 

Registration 

Introduction (Robert Graham, LeRC/John Stover) 
Delphi Round I 

Information Sources (Mark Boroush) 


9:30 

Break 


9:45 

10:00 

11:00 

11:45 

Delphi Round II 

Judgmental Methods (Mark Boroush) 

Introduction to Seminar Topic: Supersonic Cruise Technology (Charles Thomas) 
Delphi Round III 

12:00 

Lunch 


i:3G 

2:00 

2:30 

Delphi Resuits (John Stover) 

Quantitative Methods Part 1 (John Stover) Yi Workshop 
CONSENSOR Session (Charles Thomas, Mark Boroush) 
Quantitative Methods Part I (John Stover) Yx Workshop 
CONSENSOR Session (Charles Thomas, Mark Boroush) 

Yx Workshop 
Yx Workshop 

3:00 

Break 


3:15 

4:00 

5:00 

Trend Impact Analysis (John Stover) 

TIA Working Session: Four separate working groups 
Adjourn 


Day 2 



8:30 

9:00 

Dealing with Uncertainty (John Stover) 
Quantitative Methods Part II (John Stover) 
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10:00 Break 

10:15 Cross-Impact Analysis (John Stover) 

10:45 Scenarios (Charles Thomas) 

11:15 Scenario Working Session: Four separate working groups 
12:15 Lunch 

1:15 Scenario Working Session (continued) 

2:30 Scenario Results (LeRC personnel from each group) 

3:00 Break 

3:15 State-of-the-Art Technology Assessment (Charles Thomas) 

4:15 Open Discussion of Workshop Content 
4:45 Review Questionnaire/Evaluation 
5:00 Adjourn 

The contents of each major agenda item are described below. 

Introduction . In this section of the workshop we discussed the recent origins 
of futures research, tracing the early work at Rand that was conducted in an effort 
to improve military effectiveness through gaming, Delphi, and systems analysis. 
We also explained the role of normative forecasting. In addition, we set up a basic 
taxonomy for methods that were examined later in the workshop. 

Information Sources . In this section we described the range of information 
sources available to a forecaster. In particular, we described on-line data bases 
appropriate to LeRC work that are generally available today, how such data bases 
can be accessed, and the need for appropriate search terms to improve the 
efficiency of a given search (see Appendix A). 

Judgmental Methods of Forecasting . In this session we described techniques 
for collecting judgments from groups. The differences between surveys designed to 
collect judgments from large samples that replicate larger populations statisti- 
cally, and expert panels—generally small groups of experts in the subjects under 
study— were examined. Several approaches to the Delphi process were described; 
implementation through a series of questionnaires; data collection through inter- 
views; and group meetings. Discussion of the Delphi process was enhanced by 
employing a Delphi session run in the workshop as an example (see Appendix B). 


4 


The Delphi process is designed to ask a series of questions to a large group of 
experts and to strive for consensus among the experts. It has long been clear that 
bringing experts together in a conference room setting often introduces spurious 
factors in arriving at a consensus. For example, in a group setting, the strongest 
personalities rather than the strongest arguments often carry the day. Experts are 
reluctant to abandon previously published positions in front of their peers. Position 
in the organizational hierarchy sometimes inhibits free discussion of the real 
issues. To avoid these conference room inhibitions to a true debate, the Delphi 
method was developed. In this approach, the experts interact through 
questionnaires rather than face to face. In the first questionnaire, the experts 
might be asked to provide their judgment ibout the timing of a particular future 
event. The range of opinions of the experts is analyzed and, in a second 
questionnaire, those that hold extreme positions are asked to provide reasons why 
they believe that the event is likely to occur earlier or later than the group as a 
whole. These reasons are collated and fed back to the group in a third 
questionnaire. Each member of the group is asked to review his position in view of 
the reasons for the extremes and provide a new judgment, if appropriate. Over 
literally thousands of applications of this method, it has been found that groups 
generally move toward consensus. 

The two principal features of a Delphi are anonymity and feedback. 
Anonymity of the respondents (with respect to comments they provide) is 
important because it apparently permits respondents to change their minds through 
the sequence of questionnaires. The respondents receive feedback on their answers 
(and how they compare with other answers) during second and third rounds, which 
gives them extended time and more data on which to confirm or alter their original 
judgments. 

The Delphi rounds in the conference at LeRC focused on the future of 
supersonic cruise technology (SCT). The conference participants were asked to 
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evaluate the likelihood of a series of future events that could have an impact on 
the development of SCT. Supersonic cruise technology was employed as a focus for 
much of the conference activity from this point on, since it relates to much of 
LeRC's research interests. The events and their likelihoods of occurrence derived 
from the Delphi rounds were later employed in the CONSENSOR session, Trend 
Impact Analysis, and scenarios construction exercise. The five events posed to the 
conference for which participants assessed likelihood of occurrence were: 

- Event 1 : A large-scale, coordinated international terrorist attack 

results in the downing of several jumbo jets flying out of a number 
of major European airports. 

- Event 2 : The People's Republic of China, in the span of just a few 
years, becomes the U.S.'s 13th largest trading partner (similar to 
Italy today). 

- Event 3 : Stagnant growth among the Western economies and sharp 
increases in protectionist sentiments create a major international 
trade war. In consequence, the volume of world trade declines by 
more than 30 percent. 

- Event 4 : A U.S.-ASEAN* Common Market is formed, giving rise to 
rapid growth in U.S. trade and business activities with Pacific Rim 
nations. 

- Event 3 : Rapid progress in the installation of advanced communi- 

cations technologies (e.g., intercontinental fiber optic cable 
connections) makes communications with Europe and Asia as clear, 
reliable, and inexpensive as long distance connections in the domes- 
tic United States. 


Quantitative Methods of Forecasting . This portion of the workshop consisted 
of explanations and examples of forecasting using quantitative methods including 
extrapolation of time series, regression analysis, and structural modeling using 
techniques such as system dynamics. The common feature of all these methods is 
that they are extrapolative. They build their forecasts on the basis of history. As 


♦ASEAN: Thailand, Indonesia, Philippines, Malaysia, Singapore, and Brunei. 
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such, all contain the implicit assumption that forces that have shaped the past will 
continue to shape the future. This assumption is bound to be wrong in the long 
term; therefore, it is necessary to deal with uncertainty, another subject of the 
seminar (see Appendix C). 

Dealing with Uncertainty . A number of techniques exist that produce ranges 
of forecasts, rather than single values: in one approach, for example, time-series 
forecasts are amended to include the effects of future developments which, if they 
occurred, would deflect historical trends. In general, this section focused on 
quantitative techniques that utilize perceptions about future events, their probabil- 
ities, and their impacts to amend forecasts that otherwise would be deterministic. 
These techniques include: trend impact analysis, cross-impact analysis, probabilis- 
tic system dynamics, and others. We also explained risk analysis: decisionmaking 
in the presence of uncertainty. Finally, this section of the seminar examined how 
risk can be hedged using portfolio analysis techniques in which high-risk and low- 
risk items are combined (see Appendix D). 

CONSENSOR Session . The CONSENSOR is a group voting machine designed 
to be used in meetings to collect judgments of participants. In this portion of the 
session, The Futures Group used its CONSENSOR in an exercise that allowed the 
group to learn about CONSENSOR-meeting processes as well as develop group 
judgments about future events concerning supersonic cruise technology. Each 
participant in the CONSENSOR meeting has a small terminal. Each terminal has 
two dials: one dial is used to indicate judgments about a question 

posed by a moderator, and the other dial, confidence in the judgment. When all 
participants have voted, a microprocessor samples the votes and "discounts" each 
voter for lack of confidence. The discounted votes of the group are displayed on a 
video monitor in the form of a histogram. In addition, the mean vote and average 
confidence are displayed (see Appendix E). 
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Scenario Exercise . This was one of the "hands-on" workshops conducted 


during the second day of our sessions. In this portion we explained the uses and 
limitations of scenarios, and then actually constructed some alternate future 
worlds during working group meetings focused on supersonic cruise technology. We 
explained the concept of "scenario space" and discussed how quantitative 
techniques such as trend impact analyses can provide quantitative content within a 
scenario (see Appendix F). 

Trend Impact Analysis Exercise . This session demonstrated trend impact 
analysis using existing TFG software to demonstrate how time-series forecasts can 
be amended to include perceptions about future events and their impacts. Judg- 
ments concerning future events (their probability and impact) were obtained from 
the participants during the TIA working session. The original list of possible future 
events was derived from the previous Delphi rounds and CONSENSOR sessions. 
These were discussed, amended and assigned probabilities (of occurrence) and likely 
impact on the future of revenue passenger miles (as that would affect supersonic 
cruise technology). These judgments were then run on the TIA software using an 
LeRC-provided IBM- AT so that each group could see how its judgments deflected 
the "normal" extrapolative growth in revenue passenger miles (see Appendix G). 

State-of-the-Art Exercise . This technique for measuring technology state of 
the art was developed by The Futures Group under contract to the National Science 
Foundation. In this method, expert judgment is used to identify a number of 
variables and their relative contribution to the excellence of a technology. This 
technique was illustrated using data on jet engine performance in the U.S., U.K., 
and USSR. The SOA software allows analyst judgment to alter the way in which 
the excellence of technology is measured. At the session conclusion, LeRC 
participants were invited to provide their own judgments concerning various 
measurements and see the impact on the assessments produced on the computer 
screen (see Appendix H). 
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Evaluations by LeRC Participants 

At the end of the Futuring Workshop, an evaluation form was filled out by the 
participants. Below is a compiled summary of the responses. The number in each 
category represents the number of check marks given for each answer/evaluation. 
The numbers do not always sum to the number of conference participants. 
Participants left blank evaluation of those sessions for which they were not 
present. Additionally, some participants did not complete the forms in total. 


Question 1: How useful did you find the substantive information presented at the 

workshop? 


Rating 

Very 

Somewhat 

Not At All 

Useful 

Useful 

Useful 


Information Sources 

14 

28 

4 

Delphi Exercise 

31 

13 

1 

CONSENSOR Session 

31 

15 


Quantitative Analysis 

15 

25 

4 

Trend Impact Analysis 

22 

23 

2 

Cross-Impact Analysis 

IS 

26 

2 

Scenario Construction 

28 

16 

2 

State-of-the-Art 

Assessment 

11 

26 
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Question 2: If there are any areas of forecasting that you would have liked to 

see included in the workshop, please list them below. 

More technology examples 

More examples of previous forecasts and their results 

Those specific to technology planning 

Advocacy for the need for long-range planning 

Sensitivity of results ,of forecasts, to technology advances 

Introduction of bias, particularly in wording Delphi questionnaires 

More information on economic forecasting 


9 


Question 3: How would you rate the presentations made by the workshop staff in 

each of the main areas? 


Rating 


Topic 

Excellent 

Good 

Fair 

Inadequate 

Information Sources 

10 

26 

7 

2 

Delphi Exercise 

20 

23 


1 

CONSENSOR Session 

24 

20 

1 


Quantitative Methods 

11 

27 

6 

1 

Trend Impact Analysis 

22 

21 

4 


Cross-Impact Analysis 

12 

30 

2 


Scenario Construction 

15 

23 

5 

1 

State-of-the-Art 

Assessment 

6 

24 

11 

2 


Question 4: How would you rate the usefulness of the workshop's workbook as an 

initial reference for your future forecasting activities? 


Excellent 

Good 

Fair 

Inadequate 

10 

31 

5 

1 


O 
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Question 5: How effective do you feel the workshop as a whole has been in 

introducing you to the forecasting techniques appropriate to your 
work? 

Little Value Modest Value High Value 

|1 1.3 2 2.3 3 3.3 4 4.3 5 5.5 6 6.5 7 7.5 8 8.5 9 9.3 10 1 

312 122 7696611 


> 
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Question 6: Should the workshop be repeated for other members of the Lewis 

staff? 


Unqualified YES 14 


Qualified 


Total YES 


YES But only for appropriate offices/planning 
personnel 

Especially for Senior Management 
Also give a Strategic Planning Workshop 
But one week long; more selective attendance 
With more detail on mathematic techniques 
Also give special focused follow-up (e.g., T1A) 1 
But shorter 2 

33 Total: 19 


Maybe 2 


No 
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APPENDIX A - REFERENCE LIST AND SOFTWARE 


FORECASTING SOFTWARE FOR MICROCOMPUTERS 


SPRODUCT 

! OPERATING 

{ TINE 

MULTIPLE 

{GRAPHICS 

1 

PRICE '.VENDOR 

TELEPHONE 


I NODE 

SERIES 

REGRESSION 


1 

1 

1 

' 

11,2,3 Forecast 

1 

i Interactive 

Yes 

Yes 

Yes 

$90 { 1,2,3 Forecast 

(503) 585-B314 

iAREHOS/PC 

! Interactive 

Yes 

Yes 

Yes 

$1,000 iHharton Econooetric 

(215) 3B6-9000 i 

iBHDP Statistical 

lit Batch 
■Interactive 

Yes 

Yes 

Yes 

to $3,000 : 

$495 IBHDP Staticstical Software 

I 

(213) 479-7799 i 

i Software 
! CRISP 

Ik Batch 
■Interactive 


Yes 

Lieited 

to $1,500 : 

$495 iCruch Software 

(415) 564-7337 i 

lEconoaist Horkstation 

lit Batch 
•Interactive 

Yes 

Yes 

Yes 

1 

I 

$2,700 {Data Resources Inc. 

(617) 863-5100 i 

•ESP Version 2.0 

1 

l 

I Interactive 

Yes 

Yes 

Linited 

to $6,200 i 

1295 iEcononica, Inc. 

(617) 661-3260 { 

'Forecast Plus 

H Batch 
{Interactive 

Yes 

Yes 

Li ai ted 

1 

1 

$595 I Halonick Associates 

(215) 3B6-9000 ! 

IHicrostat 

It Batch 
{Interactive 

Yes 

Yes 

Li sited 

i 

■ 

$375 $ Ecosof t v Inc. 


•IficroTSP 

U Batch 
{Interactive 

Yes 

Yes 

Yes 

i 

$595 'Quantitative Hicro Software 

(714) B56-3368 ! 

IHodler Blue 

!li Batch 
'Interactive 

Yes 

Yes 

Yes 

i 

i 

$750 iftlphasetrics Corp. 

(215) 664-6480 ! 

IHodler Econooetric 

!li Batch 
{Interactive 

Yes 

Yes 

Yes 

$ 

i 

$ 4,000 : 


! Software 
IHodler HBA 

it Batch 
{Interactive 

Yes 

Yes 

Yes 

i 

i 

$1,250 : 


INHA STATPAK 

lit Batch 

{Interactive 

1 

Yes 

Yes 

Yes 

{ i 

$495 {Northwest Analytical Inc. 

1 

(503) 224-7727 ! 

IP-STAT 

1 

{Interactive 

Yes 

Yes 

Yes 

1 

$695 IP-STAT Inc. 

(609) 924-9100 ! 

ISAS/STAT Software 

ili Batch 
\ Interactive 

No 

Yes 

Yes 

1 

1 

$2,000 !SAS Institute 

(919) 467-8000 i 

ISL-HICRO 

it Batch 
{Batch 


Yes 

No 

1 

i 

$250 {Questionnaire Service Co. 

(517) 641-4428 i 

ISeart Forecasts II 

{Interactive 

Yes 

Yes 

Yes 

$695 iSaart Software Inc. 

(617) 489-2743 ! 

;spss/pc+ 

{Batch 

No 

Yes 

Lieited 

$795 {SPSS Inc. 

(312) 329-2400 i 

I St at PI an III 

{Interactive 

Yes 

Yes 

Yes 

$179 {The Futures 6roup 

(203) 633-3501 ! 

ISTAT6RAPHICS 

{Interactive 

Yes 

Yes 

Yes 

$795 !STSC Inc. 

(301) 984-5123 1 

IStatPac Sold 

{Interactive 

Yes 

Yes 

Yes 

$595 iHalonick Associates 

(215) 366-9000 ! 

IStatpro 2.0 

it Batch 
{Interactive 

Yes 

Yes 

Yes 

: i 

$795 iPenton Software Inc. 

! { 
(800) 221-3414 ! 

SYSTAT 

it. Batch 
{Interactive 

Yes 

Yes 

Yes 

. 1 
1 

$595 {SYSTAT Inc. 

(312) 864-5670 i 

The Statistician 

it Batch 
{Interactive 

Yes 

Yes 

Yes 

1 

1 

$350 {Quant Systens 

(803) 571-2B35 i 

Trend Iepact Analysis 

{Interactive 
. 1 

. t ....... . 

Yes 

Yes 

Yes 

$7,500 !The Futures 6roup 
J “ 1 

(203) 633-3501 i 
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AN ANNOTATED BIBLIOGRAPHY ON FUTURES RESEARCH 
(Including Titles on Technological Forecasting) 


Ackoff, Russell L., A Concept of Corporate Planning (NY: Wiley-Interscience, 

1970). 

Despite an abundance of literature on the subject of corporate planning, it is 
common to find confusion among managers concerning what a plan should 
contain, how planning should be conducted, and what values can be derived from 
it. Avoiding both extremes of easy platitudes and technicalities, which account 
for so much of the planning literature, an outstanding authority in the field 
explains in simple, straightforward language what should be done, who should do 
it, and why. 

Ackoff, Russell L., Redesigning the Future: A Systems Approach to Societal 

Problems (NY: Wiley-Interscience, 1974). 

Views a transition from the Machine Age to the Systems Age. The former is 
characterized by analytical thinking based on doctrines of reductionism and 
mechanics. But this is now being supplemented and partially replaced by the 
doctrines of expansionism and teleology, and a new synthetic mode of thought. 
This "postindustrial revolution" is based on automation, and its key idea is 
interaction. 

Ackoff, Russell L., The Art of Problem Solving (NY: Wiley-Interscience, 1978). 

Ackoff, Russell L., Creating the Corporate Future: A New Concept of Corporate 
Planning (NY: Wiley, 1981). 

Synthesizes three earlier volumes — A Concept of Corporate Planning (1970), 
Redesigning the Future (1974), and The Art of Problem Solving (1978)— and adds 
experience from extensive work with corporations over the last decade. The new 
concept of planning, Interactive Planning, is a systems approach that is highly 
participative. Its objective is development rather than growth, providing 
opportunities for individual and corporate development by synthesizing opera- 
tional, tactical, and strategic planning into the design of a desirable future and 
inventing ways of realizing it. 

Ackoff, Russell L., A Guide to Controlling Your Corporation's Future (NY: John 
Wiley & Sons, 1984). 

A guide to facilitate the use of interactive planning, based on Ackoff's more 
detailed book Creating the Corporate Future . 

Armstrong, J., Long Range Forecasting: From Crystal Ball to Computer (NY: 

Wiley-Interscience, 1978). 

A review of the research literature relating to predictive forecasting. 
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Ascher, William, Forecasting: An Appraisal for Policy Makers (Baltimore: Johns 
Hopkins University Press, 1978). 

This innovative study examines the records of expert forecasting of national 
trends in population, economics, energy, transportation, and technology oyer a 
50-year period. By examining forecast methods and sources, factors associated 
with greater accuracy as well as with systematic biases are identified. 

Ascher, William and William H. Overholt, Strategic Planning and Forecasting: 
Political Risk and Economic Opportunity (NY: Wiley-Inter science, 1983). 

Forecasting must be studied not in isolation, but in the context of a policymaking 
process. The authors seek to describe appropriate methods to use. 

Ayres, Robert U., Uncertain Futures: Challenges for Decision-Makers (NY: Wiley- 
Interscience, 1979). 

Provides a comprehensive global overview of current events and political and 
economic controversies, with a number of explicit forecasts for major countries, 
both developed and underdeveloped, over the next 50 years. Emphasis is given to 
the U.S., with alternative scenarios used to illustrate what might happen. 
Sociopolitical, demographic, economic and technological factors are considered, 
with the conclusion that these contemporary trends point toward major crises 
with highly uncertain outcomes. 

Baier, Kurt, and Nicholas Rescher (eds.), Values and the Future: The Impact of 

Technological Change on American Values (NY: Free Press, 1969). 

The 17 original essays are aimed "toward the discovery of ways of guiding social 
change in directions which are at the least not incompatible with the realization 
of our deepest values, and perhaps even helpful to it." (p.v) Some groundwork is 
laid for a new profession of "value impact forecasters," especially through 
methodological pieces by Rescher, Gordon, and Helmer. The other essays are 
largely focused on economics, and the editors readily confess the weakness of 
excluding views of anthropologists, sociologists, and psychologists. 

Barney, Gerald (ed.), Global 2000 Report to the President: Entering the Twenty- 
First Century (Washington, D.C.: USGPO, 1980). 

The first attempt by the U.S. government to produce an interrelated set of 
population, resource, and environmental projections, resulting in the most 
consistent set of global projections yet achieved by U.S. agencies. The Global 
2000 Study has three major underlying assumptions: (1) a general continuation 
around the world of present public policy relating to population stabilization, 
natural resource conservation, and environmental protection; (2) that rapid rates 
of technological development and adoption will continue with no serious social 
resistance; and that there will be no revolutionary advances or disastrous 
setbacks; and (3) no major disruptions of international trade as a result of war, 
political disruption, or disturbance of the international monetary systems. 

Beckwith, Burnham P., Ideas About the Future: A History of Futurism, 1794-1982 
(Palo Alto CA: B. P. Beckwith, 1984). 

One of the best ways to prepare to make scientific predictions about the future 
is to review the predictions made by the most able and/or stimulating earlier 
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futurists, and their reasons for making them. This book is largely devoted to 
critical essays on 25 writers on the future, each concluded with an assessment of 
strong points and weak points. 

Bell, Daniel (ed.), Toward the Year 2000: Work in Progress (Beacon Press, 1969). 

Essentially this is an anthology of 22 papers on specific social problems, 
sandwiched between two sections of dialogue resulting from Working Session I 
held in October 1965 (27 participants), and Working Session II held in February 
1966 (33 participants). 

Bell, Daniel, The Coming of Post-Industrial Society: A Venture in Social 

Forecasting (NY: Basic Books, 1973). ~ ' 

This widely reviewed tome is considered to be the definitive work on "Post- 
Industrial Society," and in certain respects it is. But it has many problems. It is 
a wordy assortment of essays and ideas— some fresh and challenging, some 
outdated, and some without empirical foundation despite the aura of thorough 
scholarship and Bell's reputation. There is some description, a great deal of 
covert prescription, and a few bare-knuckle assaults on "ecological radicals" and 
others such as Ivan Illich, who is seen as "a romantic Rousseauian" mouthing 
"cant words of modernity." Despite claims that the book was written in the past 
five years, the footnotes offer strong evidence of this book as a product of the 
1960s. 

Botkin, Oames W., M. Elmandjra, and M. Malitza, No Limits to Learning: Bridging 
the Human Gap (Elmsford, NY: Pergamon, 1979). 

Learning and the individual human being, rather than material resources, are the 
key to the world's future. Unfortunately, while the world has moved to a new 
level of risk and complexity, human understanding remains rooted to a world 
view that is no longer relevant. Individuals and societies will have to develop 
"innovative learning," the two chief features of which are anticipation and 
participation. 

Boucher, Wavne (ed.), The Study of the Future: An Agenda for Research 

(Washington, D.C.: USGPO, 1977). 

This volume is concerned with forecasting. In particular, it is concerned with 
the various beliefs, methods, practices, and results associated with a kind of 
forecasting that has come to be referred to in the last 10 to 15 years as "futures 
research." 

Boucher, Wayne, 3. L. Morrison, and W. L. Renfro (eds.), Applying Methods and 
Techniques of Futures Research (San Francisco: Jossey-Bass, 1983). 

Various chapters cover the evolution of futures research and its link with 
strategic planning; environmental scanning; multiple scenario analysis; and a 
non-technical introduction to futures research. 

Boulding, Kenneth E., The Meaning of the Twentieth Century: The Great 

Transition. World Perspectives, Vol. 34 (NY: Harper 3c Row, 1964). 

Portrays a future era that depends upon a stable, closed-cycle technology. 
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Bowman, dim, et al., The Far Side of the Future; Social Problems and Educational 
Reconstruction (Educational Futures, 1978). 

A forceful, innovative proposal for the reconstruction of society through 
education. The authors outline a comprehensive model for transcending tradi- 
tional education and emphasize the evolutionary changes that will help facilitate 
living and learning alternatives. 

Branch, Kristi, Guide to Social Impact Assessment (Boulder, CO: Westview, 1983). 

Seeks to provide assistance in the complex process of assessing social change, 
emphasizing an analytic approach and a theoretical framework that can be 
applied to assessing diverse events such as changes in the natural environment, 
the local economy, or the dominant technology. A major focus of the guide is 
the interrelationships that create social change. 

Brown, Harrison, The Challenge of Man's Future (NY: Viking, 1954). 

This can be considered the first genuinely futurist book. Utilizing a systems 
approach, the need for forecasting alternative futures is stressed. 

Brown, Harrison, dames Bonner, and dohn Weir, The Next Hundred Years: Man's 

Natural and Technological Resources (NY: Viking, 1957). 

Contains summaries of conferences held to discuss industrial expansion and long- 
range planning. 

Brown, Harrison, The Next 90 Years (1967). 

A pessimistic forecast. 

Brown, Harrison, The Human Future Revisited (NY: W. W. Norton & Co., 1978). 

Brown, dames, This Business of Issues: Coping with the Company's Environments 
(NY: The Conference Board, 1979). 

A work on monitoring and issues management. 

Brown, Lester, Building a Sustainable Society (NY: W. W. Norton, 1981). 

The economic consequences of the throwaway society's disregard for environ- 
mental problems are becoming more apparent. Brown contends that it is 
demand-side rather than supply-side economics that holds the key to a sustain- 
able society as world population and its inherent demand on resources continue 
to grow. 

Burchell, Robert W., and George Sternlieb (eds.), Planning Theory for the 1980s: A 
Search for Future Directions (New Brunswick, Nd: Center for Urban Policy 
Research, Rutgers U., 1978). 

Seeks to represent the current thinking on the course of future planning by the 
field's foremost experts, with 23 invited papers on physical planning, social 
planning, economic planning, macro-planning vs. local control, and what planners 
are and what they do. 
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Calder. Nigel. 19S4 and Beyond (NY: Viking, 1984). 


A look back at predictions published in 1 964 to see how good the experts in 
various fields were at forecasting the future, pointing out particular hits and 
misses. 

Cetron, Marvin, and Thomas O'Toole, Encounters with the Future: A Forecast of 
Life into the 21st Century (NY: McGraw-Hill, 1982). 

An exercise in "professional forecasting," involving the analysis and synthesis of 
data and extensive use of computers. 

Choucri, Nazli, and Thomas Robinson (eds.), Forecasting in International Relations: 
Theory, Methods, Problems, Prospects (San Francisco: W. H. Freeman, 1978). 

The first collective efforts by an international group of scholars to sketch the 
boundaries and the content of forecasting as applied to international relations. 

Clarke, Arthur C., Profiles of the Future: An Inquiry into the Limits of the 

Possible (NY: Harper <5c Row, 1964, rev. 1973). 

Offers several examples from early 20th century of forecasting failures. A 
pioneering work in the field of forecasting scientific and technological break- 
throughs. 

Commoner, Barry, The Closing Circle: Nature, Man and Technology (NY: Knopf, 
1971). 

This is a well-received popularized volume by a leading ecologist. The central 
message—that the cause of the environmental crisis is technology, rather than 
population or affluence-should not be ignored. 

Commoner, Barry, The Poverty of Power (NY: Knopf, 1976). 

Corn, 3. 3. (ed.), Imagining Tomorrow: History, Technology, and the American 

Future (Cambridge, MA: MIT Press, 1986). 

Ten original essays on America's technological expectations over the past 100 
years. Concludes that the vision of the future as a technological paradise has 
clearly been a central theme in American culture. A valuable contribution from 
the humanities, despite no mention of any recent attempts to systematically 
forecast technology and assess its impacts. 

Cornish, Edward, The Study of the Future: An Introduction . . . (World Future 

Society, 1977). 

A general introduction to futurism and future studies. Chapters discuss the 
history of the futurist movement, ways to introduce future-oriented thinking into 
organizations, the philosophical assumptions underlying studies of the future, 
methods of forecasting, current thinking about what may happen as a result of 
the current revolutionary changes in human society, etc. 

de 3ouvenel, Bertrand, The Art of Conjecture (NY: Basic Books, 1967). 

A milestone in the development of social forecasting, it laid the philosophical 
bases for forecasting methods. 
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Dickson, Paul, The Future File (Glendale, CA: Avon Books, 1978). 

An initial introduction to futurism. 

Diebold, John, Making the Future Work; Unleashing Our Powers of Innovation for 
the Decades Ahead (NY: Simon & Schuster, 1984). 

Makes the sweeping claim that "information technology is the basis of a new age 
of civilization, the Information Age." Americans may lose out to the Japanese 
and other competitors because our social, political and legal responses to the 
frentic pace of technological change are often inappropriate and almost always 
dilatory. 

Dolman, Anthony 3. (ed.), Global Planning and Resource Management: Toward 

International Decision-Making in a Divided World (Elmsford, NY: Pergamon 

Policy Studies, 1980). 

A follow-up to Reshaping the International Order: A Report to the Club of 

Rome (E. P. Dutton, Oct. 1976), which seeks to identify ways in which global 
planning and resource management capabilities could be strengthened. 

Drucker, Peter F., The Future of Industrial Man: A Conservative Approach (NY: 

John Day, 1942). 

World War II is seen as being fought for the structure of industrial society, and as 
the first war to be fought as an industrial war~where industry is the main 
fighting force itself. Drucker also feels that only the U.S. can find a 
nontotalitarian way to a free industrial society, and this cannot be through 
centralized planning. It is argued that we should plan for a great number of 
varied alternatives and build genuine local and decentralized self-government, 
avoiding both totalitarian planning and 19th century laissez-faire. 

Drucker, Peter F., America's Next Twenty Years (NY: Harper, 1955). 

Examines future developments that are already under way as a result of 
population increases. 

Drucker, Peter F., Landmarks of Tomorrow (NY: Harper, 1959). 

"At some unmarked point during the last twenty years we imperceptibly moved 
out of the Modern Age and into a new, as yet nameless, era." This is essentially 
an early description of "Post-Industrial Society." 

Drucker, Peter F., The Age of Discontinuity: Guidelines to Our Changing Society 
(NY: Harper & Row, 1969). 

Focuses on four major discontinuities: new technologies, the world economy 

(including a chapter on the Global Shopping Center), a society of large 
organizations (including a chapter on the New Pluralism), and the changed 
position and power of knowledge such that we are becoming a knowledge 
society — "the greatest of the discontinuities around us." 
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Drucker, Peter F., Managing in Turbulent Times (NY: Harper & Row, 1980). 

The three decades from the end of WWII to the mid-1970s were a period of 
regular, almost linear, economic development for most of the world's developed 
nations. In the last five years, this straightforward development has turned 
topsy-turvy. We have entered an economically turbulent time, with new 
challenges facing the managers of all types of institutions. We are also entering 
a period of turbulence in technology: a time of rapid innovation, with fast and 
radical structural shifts. 

Duignan, Peter, and Alvin Rabushka (eds.), The United States in the 1980s 
(Stanford, CA: Stanford U., Hoover Institution Publication 228, 1980). 

The 29 essays, written as unofficial background for the Republican Party, are 
arranged in two parts. Part I, "Domestic Issues," includes 14 essays on economic 
policy, government regulation, public opinions, tax and spending limits, tax 
policy, welfare reform, Social Security, energy options, environmental policy, 
health and the government, housing policy, neighborhood revitalization, and 
higher education. Part II, "Foreign Affairs," offers 15 essays on defense and 
arms control, international economics, and major nations and regions of the 
world. 

Ellul, Jacques, The Technological Society (NY: Knopf, 1964). 

Epitomizes the pessimistic climate of opinion in post-World War II Europe. 

Esfandiary, F. M., Up-Wingers (NY: John Day, 1973). 

New dimensions are seen as emerging, going far beyond right and left, conserva- 
tive and liberal, defying all the old labels. The new dimensions are up. We are 
at the beginning of the Planetary Movement and the Cosmic Upheaval. 

Etzioni, Amitai, The Active Society: A Theory of Societal and Political Processes 
(NY: The Free Press, 1968). 

A sociologist proposes a theory of societal self-control based on "valid social 
science theory" which could serve as a springboard for active participation in our 
"post-modern" society. 

Falk, Richard A., A Study of the Future Worlds: Designing the Global Community 
(NY: Free Press, 1975). 

A detailed outline of the structures and functions of new world organizations 
necessary to ensure four overriding values: the elimination of war, poverty, 

social injustice, and ecological instability. 

Feinberg, Gerald, The Prometheus Project: Mankind's Search for Long-Range 

Goals (NY: Doubleday, 1969). 

A physicist discusses the unity of mankind and the need for agreement on long- 
range goals for humanity in the light of "impending 'world-shaking' decisions." 


19 



Ferkiss, Victor, The Future of Technological Civilization (NY: George Braziller, 

1974). 

An important and well-written "exercise in utopian thinking." The four sections 
of the books are devoted to a criticism of the ruling ideology of liberalism, the 
development of a new political philosophy of ecological humanism, sketching 
some of the elements of a planetary society governed by ecological humanism, 
and describing how we will get there from here. 

« 

Ferkiss, Victor. Futurology: Promise. Performance. Prospects (Beverly Hills: Sage 
Publications, 1977). 

An informative survey regarding "Futurology"— the art and science of predicting 
the future based usually on the premise that events, both natural and social, are 
determined by impersonal natural forces acting according to observable 
scientific laws, thus making these events subject to measurement and prediction. 

Forrester, Jay W., World Dynamics (Cambridge, MA: Wright-Alien Press, 1971). 

A "preliminary effort" to model world interactions, devised for a July 1970 
meeting of the Club of Rome. The widely discussed Limits to Growth study 
followed from this work. The dynamic model described here interrelates six 
variables: population, capital investment, geographical space, natural resources, 
pollution, and food production. 

Fowles, Jib, Handbook of Futures Research (Westport, CT: Greenwood Press, 

1978). 

Futures research is a new field of endeavor, hardly more than ten years old. 
Being so young, it is more amorphous than other disciplines and enterprises. Its 
relative lack of structure, coupled with its most remarkable subject matter, can 
make futures research appear obscure and puzzling. The present volume is an 
attempt to explain the aims and content of this field. 

Freeman, Christopher, and Marie Jahoda (eds.), World Futures: The Great Debate 
(NY: Universe Books, 1978). 

Examines the alternative visions of 16 futurists and discusses critical issues such 
as food, energy, mineral resources, and policies for technical change. Concludes 
with a discussion of three world views (conservative, reformist, and radical), 
scenarios of the future from each of these three perspectives, and a chapter on 
the prospects of war. 

Fuller, R. Buckminster, Utopia or Oblivion: The Prospects for Humanity (NY: 

Bantam Books, Matrix Editions, 1969). 

Twelve papers based on talks or articles prepared over the past several years, 
providing a good overview of Fuller's thought. 

Galtung, Johan, The True Worlds: A Transnational Perspective (NY: Free Press, 
1980). 

The final volume of the Preferred Worlds for the 1990s series, sponsored by the 
Institute for World Order. 
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Gerlach, Luther P., and Virginia H. Hine, Lifeway Leap: The Dynamics of Change 
in America (Minneapolis: U. of Minnesota Press, 1973). 

An attempt by two anthropologists to supply four basic conceptual tools for 
creative participation in shaping the future: the distinction between revolution- 
ary and developmental change, and a description of systems, movements, and the 
evolutionary viewpoint. Although apparently written for high school and college 
students, and for perplexed citizens, this volume conveys some fundamental 
notions that would be useful to any professional. 

Gordon, Theodore 3., The Future (NY: St. Martin's Press, 1965). 

A popular tour of largely technological matters by a space scientist who has 
since become a professional futurist. 

Granger, C. W. 3., Forecasting in Business and Economics (NY: Academic Press, 
1980). 

A textbook addressed to future economists and managers, showing how to 
critically and quantitatively examine the future. Chapters cover basic concepts 
of forecasting, trend-fitting and forecasting, forecasting from time-series 
models, regression techniques and econometric models, use of survey regression 
techniques and econometric models, use of survey data, leading indicators, 
evaluation and combining of forecasts, population forecasting, technological 
forecasting and the Delphi method, and world models. 

Hamrin, Robert D., Managing Growth in the 1980s: Toward a New Economics (NY: 
Praeger Special Studies, 1980). 

The objective of policy should be to maximize national economic welfare, which 
must take into consideration pollution, resource depletion, the quality of work, 
income distribution, and other aspects of welfare not reflected in GNP as now 
calculated. When issues are placed in this context, the bankruptcy of the 
"growth vs. no-growth" debate is clearly seen. 

Harman, Willis, An Incomplete Guide to the Future (NY: WL W. Norton, 1979). 

After an initial chapter on the methods of futures research, Harman describes 
the breakdown of the industrial era paradigm based on scientific method and 
material progress, characteristics of the emerging transindustrial paradigm, and 
four major dilemmas that encourage the development of this new paradigm. 

Harman, Willis, and O. W. Markley (eds.), Changing Images of Man (London: 
Pergamon, 1982). 

A reprint of a 1974 SRI report. 

Harrison, Daniel, Social Forecasting Methodology: Suggestions for Research (NY: 
Russell Sage Foundation, 1976). 

The consensus of critics is that social forecasting methodology is in an 
underdeveloped state. This paper suggests a number of areas in which method- 
ological research could be usefully done. 
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Heilbroner, Robert l., The Future as History: The Historic Currents of Our Time 
and the Direction in which They Are Taking America (NY: Harper 4c Bros., 

i960]: “ 

Examines our state of mind about the future, arguing that in the past we have 
had the sustaining beliefs of optimism, but that this idea of progress "has become 
a dangerous national delusion.” 

Heilbroner, Robert L.. An Inquiry into the Human Prospect (NY: Norton, 1974). 

A bleak essay by a prominent thinker expressing little hope that social problems 
can be solved due to runaway population, the prospects of obliterative war 
(particularly resulting from blackmail by underdeveloped countries), and the 
collapse of the environment. 

Heilbroner, Robert L.. Business Civilization in Decline (NY: Norton, 1976). 

Heilbroner, Robert L., Beyond Boom and Crash (NY: W. W. Norton, 1978). 

Author of The Future as History (Harper, 1960), An Inquiry into the Human 
Prospect (Norton, 1974), and Business Civilization in Decline (Norton, 1976), 
examines fluctuations of the capitalist system. Economic planning is seen as the 
next great change that may save the system. 

Heilbroner, Robert L., An Inquiry into the Human Prospect: Updated and 

Reconsidered for the 1980s (NY: W. W. Norton, 1980). 

In this updating for the 1980s, Heilbroner adds an afterword to each of the 
original chapters. He finds that the general relaxation of mood and posture, and 
the prevailing mood of apathy, does not change the nature of the human 
prospect. 

Helmer, Olaf, Social Technology (NY: Basic Books, 1966). 

A major contribution to forecasting methodology. 

Hershel, Richard, On the Future of Social Prediction (1976). 

Hoos, Ida R., Systems Analysis in Public Policy: A Critique, Revised ed. (Berkeley, 
CA: University of California Press, 1983). 

Reprint of the 1972 edition, which reviews the ways in which systems analysis 
and the family of techniques to which it is related has created a management 
syndrome: the notion that all human affairs could be managed, and that using 
the tools of management science would solve society's problems "rationally." 

Hughes, Barry B., World Futures: A Critical Analysis of Alternatives (Baltimore, 
MD: Johns Hopkins University Press, 1985). 

This book seeks to aid those who are frustrated at the confusing array of possible 
futures, attempting to show how the bases of difference arise from alternative 
world views (or models or paradigms), competing theories and methodologies, and 
contradictory evidence or data. 
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Hussey, D. E. (ed.), The Truth About Corporate Planning: International Research 
into the Practice of Planning (Elmsford. NY: Pergamon, I9S3T 

The 30 papers deal with such topics as organized planning in major 
U.S. corporations, a survey of how planning works in 48 U.K. companies, the 
accuracy of long-range planning, pitfalls in multinational long-range planning, 
the state of the art in environmental scanning and forecasting, computer models 
for corporate planning, and a comparative study of long-range planning in Japan 
and the United States. 

Jantsch, Erich, Technological Planning and Social Futures (NY: Wiley, 1972). 

Sixteen previously published articles. In sum, "the entire process of rational 
creative action is spotlighted from forecasting and planning ... to institutional 
and instrumental change." 

Jones, Thomas, Options for the Future: A Comparative Analysis of Policy- 

Oriented Forecasts (NY: Praeger. 1980T 

Identifies and examines the underlying assumptions that led to forecasts; 
presents evidence that some images of the future are more plausible than others; 
and sets forth an intermediate position between technological-socal optimism 
and limits-to-growth pessimism. 

Jungk, Robert, and Johan Galtung (eds.), Mankind 2000 (Oslo: Universitetsforlaget, 


Contributions to the First International Futures Research Conference, held in 
Oslo in 1967. 


Kahn, Herman, On Thermonuclear War (Westport, CT: Greenwood Press, 1961 

rep. 1978). 

An analysis of nuclear strategies— should seek to prevent nuclear war as well as 
prepare to wage it. 

Kahn, Herman, and Anthony J. Wiener, The Year 2000: A Framework for 

Speculation on the Next Thirty-Three Years (NY: Macmillan. 19671. 

Considered to be a classic work by some futurists, this volume served as a 
conceptual baseline for the Commission on the Year 2000 as well as a foundation 
for continuing studies by the Hudson Institute. The final chapter on Policy 
Research and Social Change is still a valuable statement on methodology. 

Kahn, Herman, and B. Bruce-Briggs, Things to Come: Thinking About th e Seventies 
and Eighties (NY: Macmillan, 1972). 

A shorter, updated, and somewhat more popularized version of The Year 2000. 
Considerable attention is devoted to international affairs, with chapters on the 
world of the seventies and eighties, sources of stability and instability in the 
international system, military-technological possibilities, and the rise of Japan. 
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Kahn, Herman, et al., The Next Two Hundred Years (NYs William Morrow 3c 
Company, 1976). 

In this provocative work, Herman Kahn and his associates confront the issue of 
the second half of the twentieth century: whether technological and industrial 
growth will ultimately destroy mankind or bring the world peace and prosperity. 
According to the doomsayers, the population explosion, coupled with economic 
growth, could prove catastrophic for the future of humanity. Therefore, say the 
prophets of doom, economic growth must be severely limited. 

"Not so," says Herman Kahn. "Not only will the world's population increase at a 
slower rate, but the world can afford prosperity." 

Kahn, Herman, et al., World Economic Development: 1979 and Beyond (Boulder, 

CO: Westview, 1979). 

A further elaboration of the Hudson Institute world view, as previously expressed 
in The Year 2000 (1967), Things to Come (1972), and The Next 200 Years (1976). 

Kahn, Herman, The Coming Boom: Economic, Political, and Social (NY: Simon 3c 
Schuster, 1982]] ” 

A long-term perspective on mankind's economic past and present includes two 
watersheds: the agricultural revolution of some 10,009 years ago and the "Great 
Transition" in which we are living today. 

Landsberg, H. H., L. L. Fishman, and Ooseph L. Fisher, Resources in America's 
Future (Baltimore: Oohns Hopkins University Press, 1963). 

This work pioneered the extrapolation of quantified ecological trends. 

Lapp, Ralph. E., The Logarithmic Century (Englewood Cliffs, NJ: Prentice-Hall, 

1973). 

"This book is really about a conjunction of sciences — ecology and economics, or 
eco-economics." There are numerous tables to illustrate the chapters on nuclear 
power, the consumer avalanche, the knowledge explosion, the question of growth, 
and the control of technology. The potential of disaster argues for application of 
restraints on size. 

Laszlo, Ervin, A Strategy for the Future: The Systems Approach to World Order 
(NY: George Braziller, 1974). 

World order "is not merely a new 'in' term for international relations theory or 
world politics. It is rather a new conceptualization in a field that embraces 
global ecology, geopolitics, human geography, international relations theory, 
anthropology, political science, and social ethics and philosophy. It is an attempt 
to grasp the contemporary situation on this planet in all its diversity without 
erecting, or conserving, artificial frontiers either between geographic or disci- 
plinary territories." 
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Levenbach, Hans, and 3 . P. Cleary, The Beginning Forecaster: The Forecasting 

Process Through Data Analysis (Belmont, CA: Lifetime Learning Publications, 

1981). 

An introduction to the basic principles of statistical forecasting as currently 
practiced in leading corporations. Covers computer-based techniques, establish- 
ing a process for effective forecasting, analyzing data before attempting to build 
models, selecting techniques appropriate for the problem, and using 
robust/resistant methods in addition to traditional methods. 

Linstone, Harold A., Multiple Perspectives for Decision Making: Bridging the Gap 
Between Analysis and Action (NY; North-Holland, 198*0. 

Describes an original multi-perspective approach. Unlike other decision models, 
this approach combines organizational/societal, personal/individual, and 
technical/analytical perspectives. 

Linstone, Harold, and W. Simmonds (eds.), Futures Research: New Directions 

(Reading, MA: Addison-Wesley, 1977). 

The interest in this book derives not from a logical assembly of linear, 
sequential, causal chains of thought between different people, but from the fact 
that some twenty-three well-known workers in this field have arrived at very 
similar conclusions from different interests and starting points, and differing 
experiences, in a variety of countries. They agree that we must move beyond 
the objective, analytic, reductionist, number-oriented, optimizing, and fail-safe 
approach to futures problems and learn to think with equal fluency in more 
subjective, synthesizing, holistic, qualitative, option-increasing, and safe-fail 
ways. For futures research the time has come to turn in new directions! 

Lundborg, Louis B., Future Without Shock (NY: W. W. Norton, 1974). 

Lundborg advocates and foresees trends such as more social responsibility for 
business, profit optimization rather than maximization, and a flattened (more 
egalitarian) organization chart. 

Makridakis, Spyros, and Steven Wheelwright (eds.), The Handbook of Forecasting: 
A Manager's Guide (NY: Wiley-lnterscience, 1982). 

In a final summation on the future of forecasting, the editors foresee an 
increasingly important role for forecasting, but it will not become easier or 
necessarily more accurate. Business forecasting will have to be enlarged to 
include political forecasting, energy forecasting, technological forecasting, and 
related fields. Developing procedures to incorporate these other aspects into 
planning will be essential. Above all, planners and decision-makers will have to 
understand that forecasting cannot eliminate uncertainty. 

Markley, O. W., et al., Changing Images of Man (Menlo Park, CA: Stanford 

Research Institute, 1973). 

Chapters are devoted to discussing images of man in a changing society, some 
formative images of man-in-the-universe, economic man, scientific influences on 
the image of man, the characteristics of an adequate image of humankind, and 
the processes and strategies for promoting this "evolutionary transformational- 
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ist" image in contrast to the technological extrapolation^ image that we 
presently have. 

Mason, Richard, and lan Mitroff, Challenging Strategic Planning Assumptions: 
Theory, Cases, and Techniques (NY: Wiley-lnterscience, 1981). 

Chapters consider such topics as the nature of real world problems, strategic 
assumption surfacing and testing procedures, dialectical debate concepts and 
cases, policy argumentation concepts and cases, and a comparison of approaches 
to business problem solving. 

McHale, John, The Future of the Future (NY: George Braziller, 1969). 

A wide-ranging overview, aided by scores of charts and photographs, with 
particular emphasis on ecology, technology, and planetary resources. 

McHale, John, The Ecological Context (NY: George Braziller, Inc., 1970). 

Meadows, Donella H., et al., The Limits to Growth: A Report for the Club of 

Rome's Project on the Predicament of Mankind (NY: Universe Books, 1972). 

The much-publicized highly controversial, computer-based study of an MIT 
Project Team headed by Dennis L. Meadows, based on the model of Jay W. 
Forrester and embellished with 48 figures and 6 tables. 

Meadows, Dennis L., et al., Dynamics of Growth in a Finite World (Cambridge, MA: 
Wright-Alien, 1974). 

Presents the methodology behind The Limits to Growth and the details of a 
model called "World 3," a device for testing the implications of alternative 
assumptions. 

Mesarovic, Mihajlo, and Edward Pestel, Mankind at the Turning Point: The Second 
Report to the Club of Rome (NY: E. P. Dutton, 1974). 

A new and improved computer model is employed, based on multilevel hierarch- 
ical systems theory and the division of the world into ten interdependent and 
mutually interacting regions, analyzing alternative patterns of development over 
a 50-year period. 

Mesthene, Emmanuel, Technological Change: Its Impact on Man and Society 

(Cambridge: Harvard University Press, 1970). 

An overview of the now discontinued research of the Harvard Program on 
Technology and Society woven into three chapters on social change, values, and 
economic and political organization. 

Michael, Donald N., The Unprepared Society: Planning for a Precarious Future 

(NY: Basic Books, 1968). 

Chapters on the following: On the Environment of the Futurist, One Way of 

Looking at Tomorrow (focusing on complexity, turmoil, and scarcity). Three 
Prepotent Technologies (cybernation, social engineering, and biological engineer- 
ing), On the Changeover to Long-Range Planning, The Natural Resistances to 
Organizational Change, and Some Challenges to Educators. 
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Michael, Donald N., On Learning to Plan and Planning to Learn: The Social 

Psychology of Changing Toward Future-Responsive Societal Learning (San 
Francisco: Jossey-Bass, Inc., 1973). * 

Outlines a methodological planning system. 

Mitchell, Arnold, The Nine American Lifestyles: Who We Are and Where We're 

Going (NY: Macmillan, 19S3). 

Analyzes and systematizes the values and lives of Americans in such a way as to 
yield insights into why people believe and act as they do. 

Murray, Bruce C., Navigating the Future (NY: Harper & Row, 1975). 

A fairly well-balanced introduction to a number of concerns. 

Naisbitt, John, Megatrends: Ten New Directions Transforming Our Lives (NY: 

Warner Books, 19S2). *" " 

The most reliable way to anticipate the future is to understand the present. 
America is a bottom-up society, where new trends and ideas begin in cities and 
local communities. Accordingly, the Naisbitt Group, located in Washington, 
D.C., has conducted a content analysis of 6000 local newspapers over the past 
12 years. 

O'Conner, Rochelle, Planning Under Uncertainty: Multiple Scenarios and 

Contingency Planning (NY: The Conference Board. 1978). ” ~ 

An overview of corporate strategic planning. 

OECD Interfutures Group, Facing the Future: Mastering the Probable and 

Managing the Unpredictable (Paris: Organization for Economic Co-Operation 

and Development, 1979). 

A three-year study to provide OECD member governments with an assessment of 
alternative patterns of long-term world economic development, in order to 
clarify strategic policy choices in managing their own economies and in relations 
among each other and with developing countries. 

Okun, Arthur M., Equality and Efficiency: The Big Tradeoff (Washington: 

Brookings Institution, 1975). 

Concludes that "A democratic capitalist society will keep searching for better 
ways of drawing the boundary lines between the domain of rights and the domain 
of dollars." 

Peccei, Aurelio, The Chasm Ahead (NY: Macmillan, 1969). 

An Italian industrial manager assesses the macroproblems of our time, with 
particular emphasis on the growing cleavage across the Atlantic brought on by 
the technological gap. 

Polak, Fred, The Image of the Future (San Francisco: Jossey-Bass, 1973). 
Expectations of the collapse of Western civilization. 
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Porter, Allan, et al., A Guidebook to Technology Assessment and Impact Analysis 
(NY: North Holland, 1980). 

A comprehensive introduction to technology assessment (TA) and environmental 
impact analysis (El A). 

Prehoda, Robert W., Your Next Fifty Years (NY: Grosset 3c Dunlap, 1980). 

A "projected history," or extrapolated science, describing practical developments 
that will grow out of present research, integrated with social and political 
trends. 

President's Commission on National Goals, Goals for Americans (NY: American 

Assembly, 1960). 

Findings of the "opening project of the modern futures movement." 

Reich, Charles A., The Greening of America (NY: Random House, 1970). 

"By polarizing the good guys and the bad guys, and seeing no seamy side to the 
younger generation, this polemic quickly became perhaps the best-known mis- 
forecast of our time. Still, as a critique, there is a bit of baby left in the 
bathwater of overkill." 

Robertson, James, The Sane Alternative: A Choice of Futures (St. Paul, MN: 

River Basin Publishing Company, 1979). 

Author of Profit or People? (Calder 3c Boyars, 1974) and Power, Money 3c Sex 
(Marion Boyars, 1976) declares that the industrial revolution and the age of 
European ascendancy is ending and that mankind is at a turning point. 

Schmeikal, Bettina et al. (eds.), Impact of Technology on Society: A 

Documentation of Current Research (NY: Pergamon Press, 1983). 

A guide to 207 projects in ten European countries. These recent developments 
appear to amount to a second wave of the industrial revolution, with prospects of 
impact on society even greater than that of the first. 

Schwarz, Brita, U. Svedin, and B. Wittrock, Methods in Futures Studies: Problems 
and Applications (Boulder, CO: Westview, 1982). 

The authors summarize the dimensions of futures studies, and illustrate in detail 
the ways in which different methods have interacted in some real-world projects 
at the national level of policy-making in Sweden. 

Seidenberg, Roderick, Post-Historic Man: An Inquiry (Chapel Hill: University of 
North Carolina Press, 1950). 

An essay offering insight into the future phase of man's development based on 
the direction and momentum of his past. 

Seidenberg, Roderick, Anatomy of the Future (Chapel Hill: University of North 
Carolina Press, 1961). 

Continues the theme of Post-Historic Man in viewing a world moving toward 
total organization— a trend that came about unheralded and without the fanfare 
of social prophets. 
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Simon, Julian L. and Herman Kahn (eds.) The Resourceful Earth; A Response to 
Global 2000 (NY: Basil Blackwell, 1984). 

"Status of Futures Research" in Futures (August 1984), pp. 382-417. 

Five articles describing the state of futures research, uses, and new directions. 

Steiner, George, Strategic Planning: What Every Manager Must Know (NY: Free 
Press, 1979). 

A step-by-step guide in non-technical language to the fundamental concepts, 
processes, and procedures of strategic planning. Concerns topics such as 
developing corporate objectives, organizing the planning function, and evaluating 
long-range planning programs. 

Stine, G. Harry, The Hopeful Future (NY: Macmillan, 1983). 

Stine asserts as a general forecast of the future that we can do anything we want 
to do if we are willing to pay for it. A baseline scenario from the present to 
2001 is sketched, with current trends continuing essentially unchanged. 

Taylor, Gordon Rattrey, The Biological Time Bomb (NY: New American Library, 
1969). 

Predictions in the field of biology. 

Theobald, Robert, An Alternative Future for America (Chicago: Swallow, 1968, 

rev. 1970). 

Outlines changes to bring about a participatory post-industrial age. 

Theobald, Robert, Futures Conditional (Indianapolis: Bobbs-Merrill Company, 

1972). 

Collection of readings concerning the future can serve as an excellent introduc- 
tion to futurists. 

Theobald, Robert, and Stephanie Mills (eds.), The Failure of Success: Ecological 
Values vs. Economic Myths (indianapoiis: Bobbs-Merriii, i973). 

A reader of articles, cartoons, and quotations, organized in four sections: 
Economics vs. Ecologists, What Do We Mean by Development, The Search for a 
New Ethic, and Changing Personal Success Criteria. 

Theobald, Robert, Beyond Despair: Directions for America's Third Century 

(St. Paul, MN: River Basin Pub. Co., 1976, rep. 1980). 

A revised edition of Theobald's 1976 volume that has been called his most 
personal and persuasive statement. Believing unequivocally that a fundamental 
transformation is in progress, Theobald perceives American society shifting away 
from such goals as unlimited growth, top-down decision-making, and competi- 
tion, toward a more humane, more generous, and more rational society based on 
"enoughness," participatory decision-making and cooperation. 
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Thompson, Alan, Understanding Futurology: An Introduction to Futures Study 

(North Pomfret, VT: David <Jc Charles, 1979). 

An Introduction to futures studies, with brief chapters on topics such as the need 
for futures study, systems, the thermodynamic view, environmental impact, time 
and free will, wealth and growth, food, climate, education for the future, arms 
and defense, scale, and alternative technology. 

Toffler, Alvin. Future Shock (NY: Random House, 1970). 

"Like it or not, this provocative journalistic tour of the ’human side of tomorrow' 
(or, more correctly some of the human side for some people), is better known 
than any other item." 

Toffler, Alvin (ed.), The Futurists (NY: Random House, 1972). 

"The purpose of this collection is to make accessible a few of the works of the 
best-known and, at the moment, most influential futurists." 

Toffler. Alvin. The Eco-Spasm Report (NY: Bantam Books, 1975). 

"The eco-spasm or spasmodic economy describes an economy careening on the 
brink of disaster, awaiting only the random convergence of certain critical 
events that have not occurred simultaneously~so far." 

Toffler, Alvin, The Third Wave (NY: William Morrow & Co., 1980). 

We are witnessing the death of industrialism and the rise of a new civilization 
which can be more sane, sensible, sustainable, decent, and democratic than any 
we have known. 

Toffler, Alvin, Previews and Premises (NY: William Morrow, 1983). 

Based on a series of interviews with the editors of the leftist South End Press in 
Boston. The book is not a debate or an exercise in resolving differences (Toffler 
views himself as beyond the uni-dimensional right/left terminology of the 
industrial period), but an attempt to enrich past formulations in The Third Wave 
and Future Shock , extend them to new dimensions, and respond to critical 
questions rooted in the left-wing analytic perspective. 

Tugwell, Franklin (ed.), Search for Alternatives: Public Policy and the Study of the 
Future (Cambridge, MA: Winthrop Publishers, 1973). 

A collection of some of the best futurist writings in recent years. 

Twiss, Brian (ed.), Social Forecasting for Company Planning (London: Macmillan, 
1982). 

Essays here consider the concepts and techniques of social forecasting, macro- 
social forecasting and areas of ignorance, leading indicators for measuring 
change in social attitudes, discontinuities in social attitudes, designing and 
installing a system for social forecasting, forecsts of trends in the post-industrial 
society, a psycho-socio-politico-economic view of the future, and case studies of 
social forecasting. 
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Ward, Barbara, Spaceship Earth (NY: Columbia University Press, 1966). 


"Modern science and technology have created so close a network of communica- 
tion, fansport, economic interdependence— that planet earth, on its journey 
through infinity, has acquired the intimacy, the fellowship, and the vulnerability 
of a spaceship." 

Whiston, Tom (ed.), The Uses and Abuses of Forecasting (NY: Holmes ic Meier, 

1979). 

The articles here discuss specific matters such as contemporary economic 
forecasting, market forecasting in the private sector, the operations of the 
Swedish Futures Secretariat, transport forecasts that have attempted to fix the 
future, the Indicative World Plan developed by the UN, forecasts of physical 
changes in the environment, and population forecasting. 


ADDITONAL TITLES ON TECHNOLOGICAL FORECASTING 


Bright, dames, Technological Forecasting for Industry and Government: Methods 

and Applications (Englewood Clifis, Nd: Prentice-Hall, 1968). 

Bright, dames R., A Brief Introduction to Technology Forecasting: Concepts and 

Exercises , 2nd Edition (Ann Arbor, MI: Books on Demand, 1972). 

Cetron, Marvin d., Technological Forecasting: A Practical Approach (NY: Gordon 
& Breach, 1969). 

Cetron, Marvin d. and Christine A. Ralph, Industrial Applications of Technological 
Forecasting (Melbourne, FL: R.E. Krieger Publishing Company, 1983). 

Cleary, dames P. and Hans Levenbach, The Professional Forecaster: The 

Forecasting Process Through Data Analysis (Belmont, CA: Lifetime Learning, 

1982). 
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Lanford, H. W., Technological Forecasting Methodologies: A Synthesis (NY: 

AMACOM, 19721 


Linstone, H. A., and D. Sahal, Technological Substitution: Case Studies in 

Forecasting Techniques (NY: Elsevier, 1976). 


Makridakis, Spyros and Steven Wheelwright, Interactive Forecasting: Univariate 
and Multivariate Methods , 2nd Edition (Oakland, CA: Holden-Day, 1978). 

Martino, d. P., Technological Forecasting for Decision-Making, 2nd Edition (NY: 
Elsevier, 1983). 


Rosenbloom, Richard S. (ed.), Research on Technological Innovation, Management 
and Policy , Volume 2 (Greenwich, CT: dai Press, 1985). 
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Twiss, Brian C. and Henry 3ones, Forecasting Technology for Planning Decisions 
(Princeton, N3: Petrocelli Books, 1978). 

UNESCO Scientific Forecasting and Human Needs: Trends, Methods, and Message 
(Hayward, CA: Alemany Press, 1984). 

UNIDO. Industry 2000: New Perspectives (NY: Unipub, 1982). 

Wills, Gordon, et al. (eds.), Technological Forecasting and C orporate Strategy 
(Woodstock, NY: Beekman Publishers, 1969). 
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APPENDIX B - DELPHI APPROACH 


DELPHI SURVEYS 

The Futures Group has repeatedly found, over the last ten years, that much 
existing information about future technological possibilities is not published, for at 
least two highly understandable reasons. In many cases those involved are often at 
the forefront or leading edge of their fields and simply do not have time to publish 
about the status of their activities—the field is simply moving too fast. On the 
other hand, individuals and organizations often do not wish to publish since that 
would give competitors insight into their activities and, thus, would damage their 
competitive advantage. We have found that a Delphi survey is highly productive in 
uncovering and illuminating otherwise unknown and often unanticipated areas of 
technological progress and future possibilities. 

An important aspect in using the Delphi method to uncover information is the 
anonymity provided to the respondents that is offered by an impartial third-party 
practitioner. Indeed, the Delphi inquiry embodies only two basic features: 
anonymity of the inputs and feedback to the respondents. But it is the anonymity 
offered to the respondents by a knowledgeable third party that appears critical in 
unlocking otherwise closely held information. 

Before mailing a Delphi questionnaire, the information desired from the 
respondents is comprehensively thought-out, and a "protocol" or work package is 
designed and pretested. A highly important and demanding aspect of designing the 
inquiry package is to pay particular attention to ensuring that questions are as 
unambiguous as possible, and that proper ancillary material about each question 
(e.g., appropriate past trends and events and possibly important future events) is 
provided to aid the respondent's thought process. 
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Another important aspect in using the Delphi technique (in addition to the 
feature of anonymity offered to each respondent) is the ability to feed back results 
and allow the individual to reconsider his original answer based upon inputs from 
the other respondents or panelists. Delphi provides the feedback feature via a 
sequential round of questionnaires designed and processed by the practitioner. 

The Delphi technique has its own unique strengths and weaknesses. Mailed 
Delphi questionnaires can obviously be processed for large numbers of respondents. 
Nevertheless, attrition in respondents' inputs occurs during every succeeding round, 
and there is really no way of ensuring that a respondent will return his completed 
questionnaire within the desired time allocated for each round. Furthermore, with 
mailed questionnaires, a practitioner is never really sure whether the respondent or 
someone designated by him has completed the Delphi form. 

Selecting respondents and panelists is critical to a successful Delphi study. 
One must identify the potential respondents, contact them to describe the 
program, obtain their acceptance and schedule their participation. 

Identification of candidates comes from several sources: a literature search 
and review; inputs from the client; and insights and contacts of The Futures 
Group's staff. As persons are contacted they also are asked to nominate others. 
When names become repetitive, we are usually certain that the range of viewpoints 
desired will be covered (if we obtain the participation of those persons). It also 
should be noted that the viewpoints sought out are those not representing 
unanimity or consensus. Rather, The Futures Group seeks divergent views because 
the basic purpose is to obtain insights about what the future holds along with the 
reasons behind the various eventualities. (Consensus will come only if it is truly 
present not because it was designed into the study by only selecting ''compatible'' 
respondents.) 
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It should be noted that the Delphi technique is not an attempt to create a 
statistically valid sample size or conduct a large poll. It centers on obtaining 
insights from knowledgeable people. These insights and other inputs will then be 
analyzed and used to create valid forecasts. In other words, Delphi techniques are 
often referred to as attempts to create a "synthetic expert." 

As noted earlier, the task of preparing the questionnaires, workbooks and 
other material to be used in the survey is quite important. Depending on the range 
of basic areas of investigation to be covered, the questions posed and the backup or 
ancillary material for the package can often be quite extensive. It is not unusual 
for a Delphi inquiry or questionnaire to include 75-100 pages. This number can 
grow if several subjects are to be researched. In multi-subject studies, since each 
respondent or panelist will have his own area of specialty, the complete Delphi 
inquiry is not mailed to each. During the initial arrangements with him, his areas 
of expertise and the areas he desires to cover are preselected and appropriate 
material is then made available to him. 

An important aspect of the process is the designing and finalizing of the 
artifacts to be used. This requires pretesting~or in other words, the dry run of 
each. Typically this is done by employing members of the staff of The Futures 
Group and/or members of the client's organization. The design and pretest of the 
artifacts to be used is a nontrivial and demanding task. Clearly, all the artifacts to 
be used in the sequential rounds of the Delphi survey cannot be designed in its final 
form at the outset. This is the case since inputs from the preceding rounds will 
affect the designs of the later rounds. 

To provide a "feel" for what is involved in a large Delphi study, the following 
short work statement is provided: 

Accomplishing the Delphi Inquiry . It appears desirable to include at 
least 100 respondents in the mailed Delphi inquiry. Since we will be 
treating at least five basic technologies, along with the subtech- 
nologies of each, the panelists will be organized into subpanels. The 
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Delphi questionnaire will be completed, the material prepared and 
mailed to the panelists. The collating and analysis work will begin 
as the questionnaires are returned. Depending upon the specific 
design chosen, it may be appropriate to employ computer techniques 
to analyze the results, that is, to collate the median forecasts along 
with the upper and lower bounds (representing earliest and latest 
timing) and the factors associated with each of those projections. 

Since there will be many respondents involved, and since there is 
always an attrition (i.e., although persons agree, they do not always 
complete and return questionnaires during the time period 
requested), it will be necessary to maintain contact with the 
respondents to ensure that the material is obtained in a timely 
fashion and that it is comprehensive and meaningful. 

In conclusion, the Delphi survey is a valuable tool for many research topics. 
It provides rich insights concerning the state of the art and the shape of the future. 
It can be considered for many research topics, especially those where peer pressure 
or competitive intiatives constrain the flow of information. 
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APPENDIX C - QUANTITATIVE TECHNIQUES 


SIMULATION MODELING 


Bivariate Linear Regression 


Output Format 

Values of a variable (e.g., GNP) as a function of predicted values of another 
variable (e.g., energy supply). 

Input Requirements 

Paired historical values of both the predicted (or dependent) 
variable and the predictor (or independent) variable. There 
must be at least two such pairs of values, and ordinarily the 
results are valid only when there are at least ten pairs. 

Independently derived extrapolated or forecasted values of 
the independent variable; one for each time or occasion for 
which a value of the dependent variable is used. 

Forecasting Period 

Short to medium— depends upon the amount of past data and how well they 
are fitted by the regression function. It is rarely safe to forecast for more than 
half of the time range of the past data. 

Procedure 


The predicted or dependent variable (Y) is assumed to be a linear function 
of the predictor or independent variable (X). It is assumed that Y = ajX + ag, 
where aj and ao are parameters that determine the linear relationship between 
X and Y. The computational algorithm computes aj and ao from the past 
history of the paired values of X and Y. The resultant values 
of a | and ao then are used to calculate predicted values ol Y corresponding to 
the independently derived predicted or forecasted values of X. 

The most common computational algorithm uses the method of least squares; 
i.e., the values of aj and ao are chosen so that when the equation Y c = ajX = ao 
is applied to the historical values of X to compute theoretical historical values of 
Y(Y C ), the sum of the squared differences between Y c and the corresponding 
actual values of Y is minimized. An elementary application of simple differential 
calculus and solution of a simple system of two linear equations is sufficient to 
solve for the values of aj and ao that minimize the sum of the squares. 

Major Strengths 

- Is relatively simple to apply, as the technique assumes a 
simple relationship between X and Y. Since only two 
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parameters are involved, as few as two past-history X, Y 
data pairs suffice to provide a basis for prediction; and 
generally, except for assumptions about the relationship 
between X and Y that involve only one parameter, no 
other assumptions commonly give valid results with so little 
past-history data. 

Capitalizes on historical relations between the predicted 

(dependent) and predictor (independent) variable to deter- * 

mine the future values of the predicted variables, using all 
the information in the historical data pairs. 

* 

- The "goodness of fit" of the Y c to the historical Y values 
can be used to compute a measure of the strength of the 
linear relationship between the historical X, Y pairs; and 
this can be used to calculate "confidence limits," or prob- 
able upper and lower bounds for the predicted values of Y. 

In general, the closer the Y<~ to the historical Y values, 
the narrower the confidence limits, i.e., it is less likely that 
the actual values of Y will depart from the predicted 
values. The correlation coefficient is an index that can be 
used to calculate a figure of merit for the accuracy with 
which the calculated values of Y, Y c match the actual 
past-history data. The square of the correlation coefficient 
ranges from 0 through 1. A value of 0 means total 

failure of the Y c values to correspond with the 

corresponding Y values. A value of 1 for the square of 
the correlation coefficient means that the Y and Y c 
values correspond perfectly. Values between 0 and 1 may 
be interpreted as expressing the proportion of the variability 
of the historical Y values that could be accounted for by 
the calculated linear relationship between X and Y. 

Major Limitations 

The method of least squares, as commonly used, implies that 
the predicted values of the independent variable (X) are 
infallible or devoid of error uncertainty, i.e., that the only 
possible error or uncertainty is in values of the dependent 
variable (Y). Often this assumption is questionable. For 
example, the predictor variable is often forecasted in some 
fallible way. There may be past data about the Consumer 
Price Index (CPI) and the prime interest rate. A forecast of 
the future trend of CPI, then, may be used to generate a 
forecast of the future trend of the prime interest rate, using 
bivariate linear regression. But the accuracy of this fore- 
cast depends on how strongly the past values of CPI and 

prime interest rate are related and also on how accurately > 

the future trend in CPI is predicted. The latter source of 

inaccuracy is not normally taken into account in calculating 

upper and lower bounds for the forecasted values of the 

dependent variable or, more generally, in evaluating the 

accuracy of this forecating technique. 
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- The assumption that X and Y are linearly related is an 
assumption that often is dubious. The technique fails when 
the actual relationship between X and Y is grossly 
different from a linear relationship, e.g., when it is a 
parabolic relationshp with Y initially increasing with 
increasing values of X and then decreasing. Moderate 
departures from linear relationships are not uncommon and 
are often difficult to detect. They lead to corresponding 
moderate inaccuracy in the forecasts obtained. 


When the past-history data are subject to error, the effect 
of the error is to make the predicted values of Y vary less 
than they should. Values of Y that should fall below the 
mean value of Y will generally be forecasts to fall below 
that mean, but less so than they should be, and similarly for 
values that should be above the mean. The greater the 
"noise" in the past history, the greater this effect; and there 
is no way, using this method, to distinguish a weak relation- 
ship between X and Y from a strong relationship which is 
obscured by noise or error of measurement. 


The procedure, as commonly applied, assumes that all past- 
history data pairs are equally important in determining the 
values of the unknown linear-equation parameters, aj and 
ag. while the procedure can be generalized to "weight" data 
pairs, this is not the common procedure. 


The procedure is fundamentally one that generates a 
"regression forecast." The forecast of Y is made to depend 
upon a prior forecast of X. Similarly, the forecast of X 
might be made to depend upon a prior forecast of W. But 
somewhere in this series there must be a forecast that does 

nut. ucpCuu upun aiiuLiici luicta^i* v/iic way tu ur cdK tne 

chain is to have time itself as the predictor or independent 
variable. This, however, has the necessary consequence that 
the predicted (dependent) variable that is taken to depend 
only on time must either increase or decrease without limit 
with the passage of time. 


Initial Development 


Regression analysis has had a long, slow development. The correlation 
coefficient was invented by Sir Frances Galton in about 1875. Simple bivariate 
linear regression is a special case of a more general method that may involve more 
than two variables and possible nonlinear (e.g., polynomial) relationships. 
Important initial development has generally dealt with the more general cases and 
includes: 


M. S. Bartlett, "On the Theory of Statistical Regression," Proceedings of the 
Royal Society, London, A , Vol. 53 (Edinburgh, 1933), p. 260. 

R. A. Fisher, "The Goodness of Fit of Regression Formulae, and the 
Distribution of Regression Coefficients," Journal of the Roval Statistical 
Society . Vol. 85 (1922), p. 597. 
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F. Galton, Natural Inheritance (New York: MacMillan, 1894). 


K. Pearson, The Life. Letters, and Labours of Francis Galton , Vol. Ilia 
(Cambridge, Mass.: Cambridge University Press, 1940). 

K. Pearson, "Mathematical Contributions to the Theory of Evolution, III: 
Regression, Heredity, and Panmixia," Philadelphia Transcript , Vol. 187-A 
(1896), pp. 253-318. 

H. M. Walker, Studies in the History of Statistical Method with Special 
Reference to Certain Educational Problems (Baltimore, Md.: The Williams 
and Wilkins Company, 1929). 

Selected Applications 

K. Brix and E. Hansen, Dansk Nazism under Besaettelsen (Koben-Havn: 
Nordiske Studier, n.d.). 

W. C. Clark, Economic Aspects of a President's Popularity (Philadelphia, Pa.: 
University of Pennsylvania Press, 1943). 

M. Duverger, Les Partis Politiques (Paris: Armand Conlin, 1951). 

Netherlands Economics Institute, Linear Regression Analysis of Economic 
Time Series (Haarlem, 1937). 

F. A. Pearson and W. L. Myers, "Prices and Presidents," Farm Economics , 
Vol. 163 (New York: New York State College of Agriculture, 1948), pp. 4210- 
4218. 

H. W. Lanford, "Analytical Techniques: Trend Correlation Analysis," 

Technological Forecasting Methodologies: A Synthesis (New York: American 
Management Association, Inc., 1972), pp. 92-113. 

3. S. Burton and 3. R. Torh, "Forecasting Secular Trends in Long-Term 
Interest Rates," Financial Analysts 3ournal , Vol. 30 (September 1974), pp. 73- 
86 . 
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Multiple Linear Regression 


Output Format 

Values of the predicted (or dependent variable (e.g., GNP) as a function of 
predicted values of two or more other predictor (or independent) variables, one of 
which may be time. 

Input Requirements 

Sets of historical values for each of the predictor variables 
and the dependent variable for the same time period. 

- Independently derived extrapolated (or forecasted) values of 
the predictor variables for each time or occasion for which 
a value of the dependent variable is desired. 

Forecasting Period 

Same as for bivariate linear regression. 

Procedure 


The multiple regression linking the dependent variable and the predictor 
variables is a simple bivariate regression with a "best" linear combination of the 
predictors, i.e., that combination that gives the best, least-squares fit to the 
dependent variable past-history data. This best combination is found by one of 
several alternative computational algorithms, which typically involves inverting a 
matrix of order equal to the number of predictor variables. When there are more 
than three or four predictors, the task is not suitable for hand calculation. When 
the number of predictor variables is large (e.g., over 100), the task is quite onerous 
even for a large computer. The amount of computational effort required increases 
essentially in proportion to the square of the number of predictor variables. 

Major Strengths 

Strengths of bivariate linear regression also apply here. Additionally, this 
technique makes possible the determination of the relative contribution of various 
predictor variables. 

Major Limitations 

Limitations of bivariate linear regression also apply here. In addition: 

- The contribution of each predictor is its unique contribution, 
after those from all predictors that make a stronger contri- 
bution have been eliminated. Therefore, if two important 
predictors are very similar, the results may suggest that the 
one that happens to contribute slightly more is a strong 
contributor, while the other is a weak contributor. A second 
analysis, using slightly different data, might provide reverse 
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results. Thus, results of this kind of analysis tend to be 
unstable with respect to the relative importance found for 
the contributions of different predictors. 

- Therefore, results of use of this procedure do not meet a 
natural criterion for a useful statistical procedure; namely, 
they do not meet the requirement that a slight difference in 
the data will produce only a slight difference in the results. 

Initial Development 

Early uses were by psychologists applying the method to the theory of mental 
tests, where forecasts were of individual scores, e.g., college grades, not trends. In 
the mathematical development, however, it was soon recognized that the technique 
could apply equally to trends. 

- T. L. Kelley, Statistical Methods (New York: MacMillan. 

1924 ). 

Selected Applications 

This technique is popular in modern automated statistical packages (e.g., the 
Scientific Subroutine Package provided by Service Bureau Corporation). 
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Bivariate Polynomial Regression 


Output Format 

Same format as for bivariate linear regression. 

Input Requirements 

Same as for bivariate linear regression, except that the number of pairs of 
data values needed is at least one more than the degree of the polynomial. In 
practice, twice the degree of the polynomial is generally considered to be a 
practical minimum, with even more being desirable for low-degree polynomials. 

Forecasting Period 

No greater than for linear regression. In general, one more past datum is 
needed for each higher degree in the fitted function, with no corresponding 
increase in the forecasting period. 

Procedure 


Unlike bivariate linear regression, this technique does not assume that the 
variables are related by a linear expression. Rather, the relation is assumed to be 
the general polynomial: 


Y = aQ + ajX + a2X^ + a^X^ . . . 


The computational procedure is very similar to that for linear regresion, typically 
using the method of least squares. The calculations, however, tend to be more 
complex and usually require inversion of a matrix of order equal to the degree of 
the prediction equation. 


Major Strengths 

Strengths of bivariate linear regression generally apply here, except those 
that denend on the linearity assumption. Also, it is possible to uncover and make 
use of nonlinear relationships. 


Major Limitations 

Limitations of bivariate linear regression apply here, except for those that 
depend upon the linearity assumption, and 

- More data are needed. 


There are various accepted ways of establishing precedence 
for possible alternative solutions which give different sets 
of parameters, but human judgment must choose among the 
alternative methods. 
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Initial Development 


This technique has a long history, from early work on curve fitting, including 
work of Newton on the general theory of polynomial functions. The following 
references document early uses in fitting empirical time series and drawing 
inferences about the future of such series. 

J. Wishart and T. Metakides, "Orthogonal Polynomial Fitting," Biometrika , 
Vol. 40 (1953), pp. 361-369. 

M. G. Natrella, Experimental Statistics (Washington, D.C.: U.S. Government 
Printing Office, 1963), Chapter 6. 

Selected Applications 

W. W. Cooley and P. R. Lohnes, Multivariate Data Analysis (New York: John 
Wiley & Sons, 1971), pp. 85-94. 
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Linear Autoregression 
(Also called Serial Correlation) 


Output Format 

Future values of a cyclic or periodic variable (e.g., seasonal employment), if 
the past history of its fluctuations cover at least two cycles. Also provides 
measures of the strengths of possible periodic components, or higher frequency 
cycles, that may underlie past and future fluctuations in the variable. Inter- 
mediate output is a correlogram, a figure (and the corresponding function) showing 
the serial correlation for various frequencies or lags. 

Input Requirements 

Historical or past time series for the predicted variable. 

Forecasting Period 

Depends on length of past history. Never appropriate for more than length of 
past history. Safe applications are usually less than half that length. 

Procedure 


This technique involves repeated application of the method of bivariate linear 
regression in which the predictor variable is always the same as the predicted 
variable, but with a time displacement. Thus, the predicted variable is forecasted 
from its own past values with some constant "lag" in time compared to the time for 
which predictions are to be provided. For example, suppose in successive years a 
variable had values of 8, 17, 23, 25, 24, 12, 9, 2, 0, 5. With a lag of two years, the 
predictor variable value of 23 would be paired with the predicted variable value 
of 24; the predictor variable value of 9 would be paired with the predicted 
variable value of 0; etc. The method typically is applied for a number of different 
lags. With each increment of 1 in the size of the lag, the number of historical 
data pairs available for computation of the linear-equation parameters is reduced 
by 1. Consequently, the largest possible lag that may be used is one of N - 2, 
where N is the number of historical values. This technique also requires either 
that the data be in the form of a continuous historical curve or. if discrete (as is 
typically the case for futures research applications), be tabulated for constant past 
time intervals. Generalizations of the technique are possible for irregularly 
tabulated past data, but they are cumbersome and rarely used. Finally, 
considerations of result reliability usually limit the actual size of lag so as to 
provide many fewer than N - 2 different lag magnitudes. There is, however, no 
generally accepted rule for what the largest lag should be. 


Major Strengths 

Strengths of bivariate linear regression also apply here. 

Major Limitations 

- Does not easily deal with data that are not recorded for 
regular time intervals. 


45 


- Requires quite a long time series, since the greater the lag, 
the more data must be discarded in calculating the regres- 
sion. 

Initial Development 

The importance of cycles in trend data was noted by W. M. Persons in 1916. 
Early development was based on earlier work on adjusting for seasonal variations. 
For this earlier work, the cycle frequency was a priori. The major step, about 
1925, was identification of cycle lengths rather than assuming them. 

Selected Applications 

C. M. Stralkowski, Lower Order Autoregressive-Moving 
Average Stochastic Models and Their Use of the Characteri- 
zation of Abrasive Cutting Tools , unpublished doctoral 
dissertation (Madison, Wis.: University of Wisconsin, 1968). 

- G. E. P. Box and G. M. Jenkins, Time Series Analysis: 

Forecasting and Control (San Francisco, Calif.: Holden-Day, 

1970), pp. 150-158. 
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Input-Output Analysis 


Output Format 

Forecasts of the level of activity associated with each element of a system 
included in the input-output matrix. 

Input Requirements 

An input-output table specifying the amount of output of a particular setor 
that is used as an input by another sector. 

Forecasting Period 

Generally short to medium term. The main determinant is the period for 
which the input-output coefficients remain realistic. 

Procedure 


First, the input-output table must be specified. If a table for the system 
being studied is not available, a considerable amount of time and money is usually 
required to collect the necessary data and construct the table. After the table is 
in hand, it is used to trace the changes that occur in all sectors as a result of 
changes in one sector. A computer is usually used to trace how a change in the 
output of one sector affects the inputs required by other sectors, affecting the 
outputs of still other sectors, and so on. 

Major Strengths 

Input-output analysis is an extremely powerful tool for analyzing the detailed 
interactions of economic sectors. No other technique handles the interrelation- 
ships between sectors as well as input-output. 

Major Limitations 


Since for most uses the resources required to construct an input-output table 
are not available, this technique is generally only useful when input-output tables 
already exist. 

The large amounts of computation required also make this an expensive 
technique to employ. 

Initial Development 

The basic concept of input-output was developed almost 200 years ago by 
Francois Quesnay. Leon Walras developed much of the mathematics in the early 
1900s. Wassily Leontief published the first national input-output table in 1936 and 
began the modern use of input-output analysis. 
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Selected Applications 

W. Leontief, Input-Output Economics (New York: Oxford University Press, 
1936). 

H. W. Grubb, "Regional Economic Relationships Applying the Texas Input- 
Output Model to Production and Labor Employment Analyses," Report 0025- 
020-1 174-NR (Austin, Texas: Office of Information Services, Office of 
Governor, November 1974). 

C. Alnon, 3r., M. B. Buckler, L. M. Horwitz, and T. D. Reinbold, 1985: 
Interindustry Forecasts of the American Economy (Lexington, Mass.: 
Lexington Books, 1974). 
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System Dynamics 


Output Format 

Forecasts of the future behavior of a system as well as the individual 
components of that system. 

Input Requirements 

- Historical information on the behavior of the system, if 
available. 

- An understanding of the ways in which the separate compo- 
nents of the system interact. 

Forecasting Period 

Variable— depends on the system being modeled and the goals of the modeling 
effort. Most models of large socioeconomic systems are medium-to long-term, 
however. 

Procedure 

The first step is to define the desired uses of the model and to define model 
boundaries consistent with those uses. A flow diagram is then constructed that 
depicts the variables to be included in the model and the linkages among those 
variables. The relationships depicted in the flow chart are then translated to 
mathematical form, usually using the DYNAMO simulation language. These 
equations are entered into a computer ? and the mode! is tested and validated. 
Running the model into the future produces forecasts of system behavior. 

Major Strengths 

- Wide applicability . Virtually any system that involves 
feedback, is basically closed (with few exogenous influences), 
is continuous, and is nonprobabilistic is a candidate for system 
dynamics modeling. 

- Assumptions made explicit . As with all modeling techniques, 
all of the judgments and assumptions involved in the forecast 
must be made explicit and are, therefore, available for evalua- 
tion by model users. 

- Tool for experimentation . Once the model is constructed and 
tested, it can be used to test the sensitivity of forecasts to the 
different assumptions included in the model and to test the 
sensitivity of the forecasts to various policies or actions that 
might affect the system. 
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- Ease of use. A special computer language, DYNAMO, 
specially designed for system dynamics models, obviates the 
need for special computer expertise in constructing and using 
system dynamics models. 

Major Limitations 

- Models may become so large and complex that, even if they 
produce useful forecasts, it is often difficult to communicate 
the assumptions and structure of the model to others. 

Models do not include the influences of future events. Thus, 
when the model is used for forecasting, the potential impacts 
of these events are ignored. 

Initial Development 

System dynamics was developed by Prof. 3. Forrester of the Massachusetts 
Institute of Technology in the late 1950s. DYNAMO was developed in the 1960s 
and is currently maintained and improved by Pugh-Roberts Associates. (See the 
DYNAMO II User's Manual, Alexander Pugh III, MIT, Cambridge, Mass.) 

Selected Applications 

3. Forrester, Industrial Dynamics (Cambridge, Mass.: MIT Press, 1961). 

D. H. Meadows et al, The Limits to Growth (New York: Potomac Associates, 

1972). 

W. Schroeder III, Urban Dynamics in Lowell (Cambridge, Mass.: MIT System 

Dynamics Group, 1974). 


50 



KSIM 


Output Format 

Forecasts of the future behavior of a system as well as the individual 
components of that system. 

Input Requirements 

- Estimates of maximum, minimum, and current values of each 
variable. 

- Estimates of the strength and direction of interaction between 
each pair of variables. 

Forecasting Period 

Variable—depends on the system being modeled and the goals of the modeling 
effort. Most applications are likely to be medium to long term. 

Procedure 


Each variable to be studied is assigned a value between 0 and 1, where 0 
and 1 represent the limits on the variable. Interactions between variables are 
defined in terms of the strength and direction of the effect of one variable on 
another. Interactions are calculated by weighting the strength of the effect of one 
variable on another by the distance from the minimum value of the first variable. 
The interactions are solved by a computer for all variables for each solution 
interval. Sensitivity on policy runs can also be performed by changing initial 
conditions and rerunning the model. 

Major Strengths 

The simplicity of the approach makes this technique an easy one to apply. 
Most important, however, is the tact that the procedure and results can be 
communicated easily to others. 

Major Limitations 


The simplicity of the approach also leads to its major weakness~it is a 
superficial approach to modeling very complex systems. An in-depth analysis of 
system behavior is not accomplished. 

Initial Development 

KSIM was developed by Julius Kane at the University of British Columbia in 
the early 1970s. 
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Selected Applications 


3 . Kane, "A Primer for a New Cross-Impact Language—KSIM," Technological 
Forecasting and Social Change , Vol. 4 (1972), pp. 129-142. 

3 . Kane, I. Vertinsky, and W. Thompson, "Environmental Simulation and 
Policy Formulation," in A. K. Biswas (ed.), Proceedings of International 
Symposium on Modeling Techniques in Water Systems , Vol. 1 (Ottawa: 1972). 

L. G. Antle and G. P. Johnson, "Implications of OEP Energy Conservation and 
Fuel Substitution Measures on Inland Waterway Traffic," U.S. Army Corps of 
Engineers, Institute for Water Resources (March 1973). 
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DECISION MODELING 


The behavior of a large number of systems is determined, to a great extent, 
by decisions that are made by people or groups within those systems. In population 
systems the behavior of couples of child-bearing age determines the dynamics of 
the system; in market systems the collective decisions of consumers constitute 
market behavior; in industries, such as the electric utility industry, decisions by 
corporate executives on generator expansion determine many of the characteristics 
of the system behavior. Thus, in order to understand the behavior of these 
systems, it is important to understand the nature of the decisionmaking within the 
system. 

The decision model approach is an attempt to develop a model of the decision 
process applied by the decisionmakers to the important decisions within the 
system. This approach assumes that decisionmakers consider a number of different 
factors in comparing the various alternatives and that some of these factors are 
more important than others. Although a decisionmaker may not actually list these 
decision factors or consciously weigh them, they are implicit in his perception of 
the value of the different alternatives. In order to choose the "best" alternative, it 
is necessary for the decisionmaker to make a judgment about what constitutes high 
value and low value. A choice that has low cost may be considered more valuable 
than one with higher cost, while one with higher benefits may be more valuable 
than one with low benefits. How does a high-cost, high-benefit technology compare 
with a low-cost, low-benefit alternative, though? To answer this question the 
decisionmaker must specify how important cost and benefits are to him. The low- 
cost, low-benefit alternative may be perceived to be the 'tetter" alternative if 
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cost is much more important than benefits, or the reverse may be true if benefits 
are more important. 

This approach to decision modeling assumes that this process of assigning 
values to alternatives can be replicated by defining the relevant decision criteria 
and combining this information with perceptions of the importance of each criterion 
and the degree to which each alternative meets each criterion. 

The first step in such an approach is to list the decision criteria used to judge 
the alternatives. Relative importance weights are then assigned to each criterion. 
The degree to which each alternative choice meets each of the criteria is then 
estimated. The result is a matrix such as the one shown below. 


DECISION 

CRITERIA 

DECISION 

WEIGHTS 

ALTERNATIVE RATINGS 

Al 

a 2 

A3 

Aa 

a 5 

Cl 

Wj 

All 

A21 

A31 

AaI 

A51 

c 2 

W2 

Al2 

A22 

A32 

AA2 

A52 

C3 

W 3 

Al3 

A23 

A33 

AA3 

A53 

Q 

W4 

Al4 

A2A 

A34 

a AA 

A5A 
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Ci through O* represent the decision criteria. Wi through W4 are the 
importance weights for those criteria, Aj through A 5 are the available 
alternatives, and Aji through A 54 indicate the degree to which a particular 
alternative meets a particular criterion. 

Once these entries have been specified, the value of each alternative to the 
decisionmaker is computed by summing the products of the weights and ratings. 
Thus, 

Vj = 2 n W n x Ai n 

where: Vj = perceived value of alternative i 

W n = importance weight of criterion n 

Aj n = degree to which alternative i satisfies criterion n 
Since the value, Vj, calculated in this manner is dependent on the scales used 
for weighting the criteria and rating the alternatives, these values are converted to 
relative numbers by dividing by the average value. 

RVj = Vj/AV 

AV = (2jVj)/NOAA 

where: RVj = relative value of alternative i 

AV = average value of all alternatives 

NOAA = number of alternatives available. 

The alternative that has the highest relative value would be the expected 
choice for a given decisionmaker. When using this method to simulate the 
outcomes of a large number of decisions, however, the highest -valued alternative is 
not likely to be the choice in every case. In this case the importance weights and 
alternative ratings represent average values perceived by the decisionmakers. Thus 
while the highest -valued alternative should be chosen in more decisions than any 
other alternative, it may not be chosen in all decisions. (Because of differences in 
regional or individual perceptions, an alternative may have the highest value for a 
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particular individual but not for the group as a whole when average weights and 
ratings are used.) 

It is necessary, therefore, to convert the relative perceived values calculated 
for each alternative into market penetrations. This step is accomplished by 
plotting relative perceived value versus penetration for some historical period for 
which data are available. The result is a function similar to that shown below. 



Relative Perceived Value 


Once this relationship has been developed from past data, it can be used with 
future values of perceived value to determine penetration. Since there is no 
guarantee that the penetrations thus calculated will sum to 100 percent of the 
market, they must be normalized to sum to 100. 

When new alternatives that were not available historically become available 
to the decisionmaker, it is necessary only to describe these alternatives in terms of 
the decision criteria in order to introduce them into the model. For most of these 
new alternatives, however, there is a learning period involved. Decisionmakers 
often are reluctant to choose a new alternative with which they have no 
experience. They also may be uncertain of cost and other figures and thus be 
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hesitant to consider fully a new alternative. As experience with the new 
alternative begins to accumulate, however, these problems disappear. In order to 
take this learning behavior into account, it is necessary to introduce a learning 
curve adjustment that reduces the perceived value of a new alternative in the first 
years after introduction. This learning curve usually affects the new alternative 
only in the first few years after its introduction. 

This method, therefore, describes the decision process as a choice among 
competing alternatives made on the basis of how well each alternative meets 
several different criteria of varying importance. These perceptions are by no 
means static. The importance weights and alternative ratings can, and usually do, 
change with time. By incorporating this type of decision model into a description of 
the entire system surrounding the decision (possibly a simulation model of the 
entire system), a better understanding of the behavior of the system should result. 
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APPENDIX D - MODIFYING EXTRAPOLATION WITH JUDGMENT 
(Dealing with Uncertainty) 


fr/&/ 


A DISCUSSION OF SOME METHODS OF TECHNOLOGY ASSESSMENT 


A technology assessment centers around a problem or a technology, 
either of which may be prospective or current, social or scientific. An 
assessment usually attempts to answer not only what the outlook is, who 
will be affected and how, but also includes an analysis of how policies 
which may be followed can influence the further development of the 
technology or the problems and its effects. 

There is no codified set of techniques, no simple crank to turn to 

satisfy these objectives. The methods of technology assessment which 

have been used in past studies have been drawn from many different 

disciplines, including, most prominently, economics, social psychology, 

statistics, and operations analysis. The choice of the appropriate 

technique depends on the judgment of the researcher as well as the 

nature of the problem and the level of funding of the project. As 

Joseph F. Coates of the National Science Foundation recently put it: 

Technology assessment is an art form... It is separate, 
distinct, different from (existing disciplines) and as 
is any other art form, implies both a degree of creativity 
as well as the use of techniques. It also implies that 
not everyone is talented in undertaking or understanding 
the art form.* 

Martin V. Jones, in an early set of technology assessments performed 
for the Office of Science and Technology, defined seven major steps in 

*Joseph F. Coates, "Institutional and Technical Problems in Risk 
Analysis," from Proceedings of a conference on Bulk Transportation of 
Hazardous Materials By Water in the Future, sponsored by the Committee 
on Hazardous Materials, July 9-10, 1973, College Park, Maryland (here- 
after called Joseph F. Coates) . 
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making a technology assessment. These steps included: 

1. Defining the assessment task, in which relevant issues 
and major problems are discussed, project ground rules 
established, and the scope of the inquiry is set. 

2. Describing relevant technologies including not only 
the technology being assessed, but other technologies 
supporting or competing with the technology under study. 

3. Developing state of society assumptions, involving the 
identification and description of major non-technical 
factors which could influence the application of the 
technology. 

4. Identify the impact areas, that is, societal character- 
istics that will be most influenced by the technology. 

5. Making preliminary impact analysis: tracing and inte- 
grating the process by which the assessed technology 
makes its societal influence felt. 

6. Identifying possible action options through the 
development and analysis of various programs for 
obtaining public advantage of the technology. 

7. Completing the impact analysis through study of the 
ability of each action option to alter the previously 
established societal impacts.* 

Most technology assessments go somewhat further than these seven 
steps. The identification of interest groups is an important inquiry in 
most studies. It is not simply enough to identify impact areas; the 
questions of who is affected, when, and how must also be addressed. In 
addition, the study of action options and state of society assumptions 
must include macrolevel alternatives to the technology under study. A 
technology assessment of solar energy, for example, should probe not only 
the means for producing and using solar energy and the effect of such 


*Martin V. Jones, A Technology Assessment Methodology , MITRE MTR6009, 
7 Volumes, (McLean, Virginia: The Mitre Corporation) 1971 
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technologies on society, but also whether solar energy or some other 
energy form is more appropriate, and the consequences of reducing energy 
demand . 

Given the objectives of technology assessment and the major elements 
of such studies, a logic flow chart such as that shown in Figure 1 can 
be prepared. This chart was derived from a more elaborate logic flow 
being used in a study conducted by The Futures Group under contract to 
the National Science Foundation in the field of geothermal energy.* One 
of the first things apparent in this flow chart is that several iterative 
loops are involved (Joseph F. Coates suggests that a technology assess- 
ment should really be done three times: "the first time to define and 
understand the problem, the second time to do it right, and the third to 
burnish the results, fill in the details, and to bring the report to the 
best possible state within the available time, budget, and manpower." 

Beginning at the left of the diagram: 

Task 1. State of the Art . The state of the art is defined first. 

In this task, the major question is, "What do we know about the technology 
or problem under study?" 

Task 2. Other Conditions . Here the focus is on other aspects of 
change which impinge on the subject under investigation. For example, 
this task may address concomittant technologies which, if successful, 
would limit or in some way amplify the basic technology. In addition, 
this task might center on external social conditions which could change 
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TECHNOLOGY ASSESSMENT LOGIC FLOW 








the impacts of the technology or the values by which the impacts are to 
be judged. 

Task 3. Forecast . This task involves generating an image or set 
of images of the future of the technology under study. How might it 
evolve? What are the most important breakthroughs? When might they 
occur? What paces them? 

Task 4. Policies for Modifying the Technology or Problem . In this 
task, policies are nominated and defined which might change the time 
scale or nature of the technology. Typically, the definition of a policy 
might include identification of the agency which could perform the 
action, the cost of the policy, its timing, and its intended conse- 
quences. Note that the output of Task 4 provides an input to Task 3; 
that is, the policies generated here are designed to amend the techno- 
logical forecast. 

Task 3. Scenarios . In this task, forecasts are grouped into 
self-consistent sets, alternative futures depicting the evolution of 
the technology or problem under study in the presence of policies con- 
sidered in Task 4. In effect, the output of Task 5 is a statement of 
what the world might be like in the particular domain of the assessment. 
The remainder of the analysis is devoted to an identification of the 
consequences of these alternative worlds. 

Task 6. Interest Groups . Here the groups likely to be effected 
by the emerging technology or problem are identified. 

Task 7. Impact Description . Given the interest groups and the 
alternative futures produced in Tasks 6 and 5 respectively, this task 
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defines and estimates specific impacts. Thus, at this point in the 
study, an array can be constructed of interest groups, how they are 
ikely to be affected in the various worlds depicted by the scenarios, 
each scenario having been determined by a particular technological or 
problem-oriented policy set. 

Task 8. Assessment . Now the problem is : are the forecasted impacts 

likely to be seen as good or bad? Of course, such judgments are value- 
laden and multidimensional. An impact seen as good for one group might 
be bad for another. Furthermore, "good" and "bad" are relative terms. 

Yet despite these difficulties, the assessment accomplished in this task 
necessarily must deal with these issues. 

Task 9. Modifying the Impacts . The object of Task 9 is to invent 
policies which seem likely to improve the future state of affairs, that 
is, the conditions identified in Task 8. Nominated policies are fed 
back to Tasks 7 and 3. The loop to Task 7 is designed to test policies 
which have as their major focus the modification of impacts; the loop 
to Task 3 is designed to test policies which are designed to modify the 
technology itself. When policies have been found which produce the 
"best" world for the interest groups, they form the basis for action 
recommendations . 

What methods are available to accomplish the objectives of these 
tasks? Figure 2 summarizes some which have been used in technology 
assessment. The basis for this chart is the comparative analysis of 
thirteen previous technology assessment performed by Martin V. Jones 
and experience with three additional studies conducted by The Futures 
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Group.* The "preferred" entries are obviously a matter of personal taste and 
bias and depend greatly on the case at hand. Other people would undoubtedly 
add other methods and designate different techniques as more useful than those 
snown. Nevertheless, the chart illustrates the wide range of interdisciplinary 
tools which are available to the assessor. 

Several of the methods listed on this chart may be unfamiliar. These 
approaches are discussed in the appendicies of this report. 


*Martin V. Jones, A Comparative State-of-the-Art Review of Selected U.S. 
Technology Assessment Studies, (McLean, Virginia: The Mitre Corporation) 

May 1973. The Futures Group's studies include: "A Technology Assessment of 

Geothermal Energy," for the National Science Foundation; "U.S. Ocean Shipping 
Technology," for the U.S. Department of Commerce, Maritime Administration; and 
a proprietary study in the field of energy. 



PROBABILISTIC SYSTEM DYNAMICS 


Introduction 


Modeling techniques are widely used to predict the behavior of systems. 
Modeling is an exact science when applied to physical systems such as planetary 
motion. It also has been shown to be a useful tool in the understanding and 
prediction of the behavior of economic systems. Such models, however, are not 
entirely accurate. Applied to social systems, such models usually are even less 
accurate in predicting specific values, but are useful in displaying the basic 
behavior of the system under study. 

A mathematical model is a detailed mathematical description of the relation- 
ships among the most important variables within some system. Thus, given values 
of these variables at one point in time (the condition of the system at that time), 
the state of the system at some later time can be calculated. The utility of 
mathematical models arises from the basic assumption that human judgment is not 
capable of reliably determining how complex systems will change with time. 
Although a person may have detailed knowledge about the parts of a system, the 
number of interactions taking place usually is too large for the human mind to 
handle. Thus, approximation, reduction of considered variables, and intuition are 
all used to produce a final judgment. Mathematical modeling assumes that by using 
human judgment to specify the parts of the system and how they interact 
separately (a simpler task with fewer variables involved), a model can be 
constructed that will display the behavior of the system in a more reliable manner. 
This model then can be used, through experimentation, to come to a better 
understanding of the system. 

Probabilistic system dynamics (PSD) is a method of mathematical simulation 
that combines the system modeling concepts of system dynamics with the 
probabilistic event interaction techniques of cross-impact analysis. 


System Dynamics 

System dynamics was developed in the 1950s by Professor Jay Forrester of 
the Massachusetts Institute of Technology. 1 The original concepts were derived 
from servomechanism analysis in which systems are depicted as a series of 
interlocking feedback loops. 

This simple feedback loop is basic to the theory of system dynamics 
modeling. Often many simple loops interact with one another, producing complex 
systems. Figure 1 depicts two of the feedback loops in a model of the Japanese 
economy developed by The Futures Group. In the positive feedback loop, 
increasing industrial production increases demand for energy. This increased 
demand normally will lead to increased supplies and increased stocks of energy 
supplies. These increased supplies will, in turn, allow increased industrial produc- 
tion. This is the growth loop and will lead to unrestrained growth unless a negative 
loop also is present. In the negative loop increased industrial production leads to 
increased energy usage which decreases the stock of energy supplies. 
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Figure 1. Energy Feedback Loops in the Japan Model 

A second crucial point in this model of the decision-making process is the 
effect of delays. Often the decision-maker does not have up-to-date information 
about his environment. He is, therefore, making decisions on the basis of what the 
situation used to be, rather than what it now is, or is taking action that will not 
translate into concrete changes until some future time. The example of a 
television cameraman can be used to illustrate this point. Consider the technician 
at the 3ohnson Space Center in Houston who had the job of controlling the 
television camera of the lunar rover on the moon as it filmed the takeoff of the 
landing module. His job was to keep the picture of the landing module in the 
center of the screen as the module began the return trip to earth. The 
environment for his decisions (how to tilt the camera) was, of course, the picture 
he saw on the screen. However, due to the time necessary for the picture to travel 
from the moon to the earth he was seeing the scene as it had been a few seconds 
earlier. Furthermore, it took another few seconds for his adjustments to reach the 
moon and cause the camera to move. Thus, if he saw the lunar module moving out 
of the picture he would start to adjust the camera immediately, but would not 
know whether his adjustments had been too much or too little for several seconds. 
The fact that he handled his task so well is due mainly to his advance knowledge of 
the exact lift-off time and speed of the module. In most situations, such exact 
advance knowledge is not available. Thus, delays are an important consideration in 
decision modeling. 

When system dynamics modeling is applied to a specific problem, the first 
task is to identify the feedback loops that appear to have bearing on the problem. 
The next step is to study these feedback loops to determine the decision points and, 
once these are identified, to describe the decision process, including the informa- 
tion used in these decisions. Building on this information a simulation model is 
built and tested by comparing its behavior with actual system behavior. The model 
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may be refined until it appears to accurately represent the real system. This 
model then is used as a tool for experimentation. Changes are made in the model 
in an effort to find methods of improving the system's behavior. These changes can 
then be instituted to the real system. It should be noted that this improvement of 
the real system is the ultimate goal of model building. The goal is not to forecast 
what future conditions actually will be, but rather to show how the system will 
behave given a certain structure. That knowledge should lead to an improvement 
of the system and thus invalidate the model "forecast." 

The theory of system dynamics modeling appears to have changed very little 
from its first applications in the late 1950s. Emphasis has, until now, been on new 
applications of the technique rather than on improvements to the technique itself. 
The widespread use of the technique can be credited to its four major strengths. 
The first of these is the wide applicability of the technique. Feedback processes 
are a part of practically every decision system. As long as the system involves 
decision making (and, therefore, feedback loops), is basically closed (that is, with 
few important exogenous influences), is continuous and nonprobabilistic in nature, 
it is a candidate for system dynamics modeling. 

The second major strength is one common to all analytical techniques. The 
judgments and assumptions involved in the analysis are made explicit. Almost any 
detailed study of a problem includes some real knowledge and hard data along with 
many assumptions and estimates. In many cases, it is hard to separate these 
components and even harder to determine the importance of each to the final 
conclusion. A mathematical model is explicit in terms of how the final results are 
determined. Anyone can trace the process, given the model. In discussing the 
system behavior described by the model, everyone is working from the same base 
point, the same set of assumptions. While everyone may not agree about the 
assumptions in the model, they are, at least, clearly visible and can be modified or 
discarded as necessary. By reducing the problem from one of estimating the total 
system behavior (as in mental models) to one of estimating only those parts of the 
system that are not known, a more reliable model should result. 

The third advantage is that, once the model assumptions are accepted, the 
model can be used as a tool for experimentation. Experimentation with a 
mathematical model is usually much less costly and less risky than experimentation 
with the actual system. The advantage of a model as an experimental tool lies only 
partly in the ability to make a change in the model and observe the resulting 
behavior. The real power of such a tool is that the behavior resulting from an 
experimental change need not be believed as an act of faith. Results can be traced 
through the system and exact causes can be identified. Thus, if a certain change 
produces a counterintuitive result, the causes of that result can be isolated. They 
may prove to be errors in the model, in which case the errors are rectified, or they 
may prove to be real, in which case new knowledge is gained. In either case 
counterintuitive results cannot be lightly dismissed as obviously wrong or 
ridiculous. 

The fourth advantage is ease of use. A special computer language and 
compiler, DYNAMO, has been developed for system dynamics models. DYNAMO 
accepts inputs in the form of normal mathematical equations, provides extensive 
error checking, and automatically formats the outputs. Thus, no extensive 
computer training is required. 
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Although system dynamics is a powerful technique and has been applied to 
many important problems, there are important drawbacks to the technique as well. 
System dynamics models do not account for probabilistic future events. In many 
systems the decision environment is directly influenced by discrete events that 
may or may not occur. The decision by Arabian oil-producing countries to cut oil 
production in late 1973 certainly had significant influences on the economic 
systems of many countries. Yet such an event would not be a part of that system. 
Events of this type—though exogenous to the model— may well be the most 
important influences on the system over time. 

Another limitation is that model structure remains essentially fixed with 
time. Relationships among model variables are detailed once in the model and do 
not change. While this may not be a problem for simple systems and short time 
spans, models of the Limits to Growth type suffer from this rigidity. Relationships 
among model variables may change because of causal forces within the system 
because of events occurring within or outside the system. The changes should be a 
part of the dynamics of the system. 


Probabilistic System Dynamics 

Probabilistic system dynamics uses the concepts of cross-impact analysis to 
add the expected impacts of future events to system models. As an example of the 
influence of events on model structure, consider the energy feedback loops 
discussed in the last section and presented again in Figure 5.2. In this figure, 
however, new loops have been added (dotted lines) that correspond to the impacts 
of events. The energy supply fraction represents that fraction of demand that is 
filled by available supplies. Although abundant energy supplies have kept this 
fraction equal to or greater than one in the past, future energy shortages may well 
cause it to fail below one. 
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ties of various events are affected. Specifically, the probabilities are increased for 
two important energy events (that Japan and the USSR engage in the large-scale 
development of Siberia and that fusion power is developed and is implemented in 
Japan). The second event upon occurrence increases the supply of atomic power, 
thus increasing domestic energy supplies. The first event, upon occurrence, will 
add significant new supplies of natural gas to Japan's pnerpy picture (natural °as 
supplies today are an insignificant energy source for Japan!." Thus, not only is a 
new fuel added to energy supply (as well as all the important price and fuel trade- 
offs that are a part of that new fuel), but imports, balance of payments, and 
exports may be affected as well, depending on the exact nature of the trade 
agreements. These events may also affect other events in the cross-impact matrix 
that will, in turn, influence the model in other ways. 


Of course, with only one or two events, probabilistic calculations have little 
real meaning. Fusion power will either become an alternative energy source or it 
will not. To say that atomic power will supply 25 percent of all energy needs in 
Japan because fusion power has a 50 percent probability of being developed, would 
be absurd. Rather, we should say that atomic power will provide 35 percent of 
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Figure 2. Energy Feedback Loops in the 3apan Model 

energy needs if fusion power is developed, and 15 percent if it is not. However, if 
our real concern is not the percentage of energy supplied by atomic sources but the 
total energy availability in 3apan, then probabilistic considerations begin to make 
sense if we include a few more events in the model. Suppose we included events 
concerning the development of needed technology for oil shale, tar sands, solar 
electricity, geothermal drilling, etc. We may not be able to say with certainty that 
any of these developments will or will not take place, but we can say that some of 
them will probably happen while others will not. By combining the probabilities of 
occurrence of these events with their expected impacts, we can develop a model 
that will indicate the expected future supply of energy in 3apan. It will not 
determine which events actually occur or what the real fuel-mix will be (as this 
will depend on actual event occurrences), but it can give a better indication of the 
total energy supply than if these events had been entirely omitted. Furthermore, 
the effects of policy decisions which can influence the development of different 
energy technologies included in the model (geothermal hot rock drilling, for 
example) can be displayed in terms of the expected influence on the total supply of 
energy. It is here that the main contribution of PSD lies. 

In order to develop a PSD model of a system, two processes are followed 
simultaneously: the development of a basic system dynamics model and the 

identification of the elements of the cross-impact matrix. The identification of 
the event set and the estimation of probabilities and impacts can also serve as an 
excellent first step in the integration of decision-makers into the modeling effort. 
The Delphi approach to information gathering may, in many cases, be applicable. 

Responses of a panel of experts (either to questionnaires or personal 
interviews) are used to define the set of events important to the study, the impacts 
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of these events on the model, the impacts of the events on each other, and the 
probability versus time characteristics of each event. The experts may also 
contribute initial information regarding the structure of the model itself. These 
questionnaires or interviews provide not only expert judgment in the areas already 
mentioned, but also indicate those areas in which there is not good agreement 
about present relationships or future developments. These areas, then, can become 
the subject of increased attention when the model sensitivity and policy studies are 
conducted. 

Using the information supplied by the experts, a system dynamics model, 
showing the variables important to the area of study and relationships among those 
variables, and a cross-impact matrix, including the events, their probabilities and 
impacts, are constructed. 

The first step in the development of the cross-impact matrix involves 
calculating the time dependent probabilities of each of the events. Each 
respondent supplies estimates of cumulative probabilities for two years for each of 
the events. By fitting an S-shaped curve to these two points a curve is developed 
describing the cumulative probability of each event versus time. By combining the 
curves from all the respondents for each event, the final curve is obtained. If 
there is a large amount of disagreement among the experts on any particular curve 
this result would, typically, be reported back to the respondents for further 
consideration. 

The impacts of events on events are then described. For example, if one 
event is that a new method of reclaiming and recycling scrap zinc at one-third the 
cost of new zinc production is deveioped this event might originaiiy nave a 
probability of occurring by 1985 of .15. However, if another event, that proven 
world reserves of zinc quadruple through new' discoveries, were assumed to occur, 
it would have substantial impact on the probability of the first event, possibly 
reducing its probability by 1985 from .15 to .05. 

The impacts of the events on the model are then described. These impacts 
may come from expert judgment in cases where the impact is uncertain or they 
may be calculated when the impact is obvious and not judgmental. These impacts 
may take several forms. The actual structure of the model may change, 
coefficients of the equations may vary, new terms may be added to or subtracted 
from equations, etc. 

Some variables in the model will also have impact on the event probabilities. 
These impacts are also estimated and included in the cross-impact matrix. As an 
example, assume that the supply of zinc to some country is a factor in an economic 
model of that country. The supply of zinc would undoubtedly depend on domestic 
demand for zinc, world demand for zinc (competing with domestic demand), 
domestic supplies of zinc ore, world supplies of zinc ore, and the price of zinc. 
However, if the event discussed earlier concerning the development of a low-cost 
zinc recycling technique were to occur, the above relationships concerning zinc 
supply would have to be altered. Zinc could now also be supplied through scrap 
recycling (an addition to the supply equation), and domestic and world stocks of 
products containing zinc and their obsolescence rates now become important as 
well. This illustrates one type of impact of an event on a model. In turn, model 
variables, the shortage of zinc, for example, would affect the probability of the 
events (by causing increasing exploration for new resources and increasing expendi- 
tures for new recycling techniques). 
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A run of an entire model constructed using probabilistic system dynamics 
involves the simultaneous operation of a DYNAMO computer program (the basic 
model) and a FORTRAN IV program (the cross-impact sector). The computational 
sequence consists of the following steps: 

1. Initial values for the model variables are used to compute 
new values for those variables for the first solutuion 
interval (a year or part of a year). 

2. These new values are transferred to the cross-impact 
matrix. 

3. New values of event probabilities are determined from the 
original values and the impacts of the model variables. 

4. The expected impacts of events on events are calculated 
using the probabilities of the events to weight the occur- 
rence and nonoccurrence impacts. 

5. The probabilities from the cross-impact sector are 
transferred back to the basic model, event impacts on the 
model are calculated, and the basic model is revised 
accordingly. 

6. New values for the model variables are calculated for the 
next solution interval using the model as revised by the 
impacts from Step 5. 

7. Steps 2 through 6 are repeated until the end of the time 
span under study. 

Sensitivity analyses can be run to test the sensitivity of the model to the event 
probabilities, impact estimates and model assumptions. 

Policy tests can be run by deciding how specific policies would change the 
model (event probabilities, model structure, impacts), making the appropriate 
changes and rerunning the model. 

The main advantages of probabilistic system dynamics are: 

1. The inclusion of events in the model allows occurrences 
outside the area of focus of the basic model to be taken 
into account in the model predictions. In this manner, any 
number of exogenous events may be included. Thus, the 
scope of the model is no longer limited strictly to the 
closed system defined by the system boundaries. 

2. The structure of the basic model itself becomes dynamic. 
Relationships among variables and even among sectors of 
the model may change with time as the impacts from other 
parts of the model and from the events accumulate. 

3. Policies can be tested in terms of their effects on the 
relationships among model variables, model structure or 
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event probabilities. Policies that have their main impact 
on areas outside the basic model can still be tested for 
impact through the effect of those policies on exogenous 
events. Thus, a much more complete description of the 
side effects of policy decisions is possible. 


A Simple Example of Probabilistic System Dynamics 

Figure 3 depicts a simple system dynamics model of an electric utility with 
events included. In this model, projected consumption of electricity and the 
average load factor determine the projected peak demand. This leads to a 
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Figure 3. Simplified Probabilistic System Dynamics Model of an Electric Utility 
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determination of total capacity required, which, along with current capacity, 
determines the amount of new capacity that needs to be added. These additional 
requirements are allocated, on the basis of several decision criteria, to oil-fired, 
coal-fired, and nuclear plants. Orders are placed for these plants and, after 
planning and construction delays, new plants come on-line. After some period of 
time these plants wear out or become obsolete and are retired. 

The dotted lines show some of the impacts of the events on the model. The 
strip mining bill will affect the cost of coal and possibly the availability as well, 
thus affecting coal's ability to compete for new capacity. Another oil embargo 
would affect the present and the perceived future availability of petroleum. A 
nuclear moratorium would remove the nuclear option from capacity expansion. 
Finally, the implementation of peak load pricing would shift the load curve and 
affect peak demand. As an example of these impacts, assume that the cost of coal 
generation (COAL$) is given in this model as a TABLE function of time.* 

A COAL$ • K = TABLE (TCOAL$, TIME • K, 1975, 2000, 5) 

T TCOAL$ = 10/13/16/19/21/25 

With the strip mining event included in the model this TABLE function might 
be rewritten as 

A COAL$ • K = TABLE (TCOAL$, TIME • K, 1975, 2000, 5)*(1 + EL • K*CPI) 

T TCOAL$ = 10/13/16/19/21/25 

where EL • K is equal to zero if the event has not occurred and to 1.0 if it has 
occurred. CPI would be equal to the percentage increase in the cost of generation 
if the event occurs. 

As a second example, the fraction of new generation allotted to coal might 
be a function of the costs and other characteristics of coal, oil, and nuclear plants: 

COAL • K = f (COAL$, OIL$, NUC$, CRA, OFA, NFA...) 

where this function actually represents a series of equations that compare 
characteristics and allocate new capacity. With the inclusion of events in the 
model, this part of the model might be expressed as: 

COAL • K = (1 - E3 • K)*f(COAL$, 01L$, NUC$, CFA, OFA, NFA,....) + 

E3 • K*f(COAL$, OIL$, CFA, OFA,....) 

where E3 • K equals zero when the nuclear moratorium event has not occurred and 
1.0 when it has occurred. 

The impacts of model variables on events are shown in Figure 3 as dashed 
lines. The amount of electricity generated by coal, and therefore, the demand for 
coal, might affect the probability of strip mining legislation. Similarly, the amount 


♦The equations are written in DYNAMO, the computer language developed by 
3. Forrester for System Dynamics. (3. Forrester, Industrial Dynamics, MIT Press, 
Cambridge, MA, 1961.) 
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of oil generation might have some effect on the probability of an embargo, and the 
amount of generation by nuclear plants might affect the likelihood of a nuclear 
moratorium. The average load factor would have an influence on the likelihood of 
peak load pricing. The impact of nuclear capacity on a moratorium might be 
expressed in equation form as follows: 

A INUC3 • K = TABLE (T1NUC3, NUC% • K, .10, .40, .15) 

T TINUC3 = .30/1.0/2.4 

where INUC3 • K is the impact (in terms of odds ratios) of nuclear capacity on 
event 3 (nuclear moratorium) and NUC% • K is the percentage of total capacity 
that is nuclear. 

Finally, the impacts of events on events can be handled in a number of ways 
depending on the calculational procedure used for the cross impacts. In some cases 
FORTRAN programs may be written to handle these calculations and link with the 
DNYAMO model. In the simplest approach, however, the equations can be written 
in DYNAMO. For example, if the event impacts are as shown in the example in 
Figure 1, the equations for event 3, the nuclear moratorium, might be written as 
follows: 

A PROB3 • K = ODDS3 • K/(l + ODDS3 • K) (1) 

PROB3 = Probability of event 3 
ODDS3 = Odds of event 3 

A ODDS3 • K = IODDS3 • K*123 • K*INUC3 • K (2) 

ODDS3 = Odds of event 3 
IODDS3 = Initial odds of event 3 

123 = Impact of event 2 on event 3 
INUC3 = Impact of the amount of nuclear 
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A IODDS3 • K = IPROB3 • K/(l - IPROB3 • K) (3) 

A IPROB3 • K = TABLE (IPROB3, TIME • K, 1975, 2000, 5) (4) 

T TIPROB3 = 0/.05/.10/.15/.18/.20 (5) 

IODDS3 = Initial odds of event 3 
IPROB3 = Initial probability of event 3 

A 123 • K = FIFGE (.47, 1.0, E2, JK, .1) (6) 


R E2 • KL = MAX (E2 • 3K, FIFGE (1.0, 0, PROB2 • K, .OL) (7) 

123 = Impact of event 2 on event 3 
E2 = Indicator of occurrence or nonoccurrence 
of event 2 

PROB2 = Probability of event 2 
OL = Occurrence cut-off level 

In Equation 1 the probability of event 3 is calculated from its odds. The odds 
are calculated as the initial odds times the impact odds ratios~in this case, one 
odds ratio for the impact of event 2 on event 3 and one for the impact of a model 
variable (the percentage of capacity that is nuclear) on event 3. The initial odds 
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are calcualted from the initial probability of the event (Equation 3) which is given 
in a TABLE function. The impact of event 2 on event 3 is set equal to 1.0 if 
event 2 has not occurred or to the odds ratio expressing the effect of event 2 on 
event 3 (.47 in this example) if event 2 has occurred. The occurrence of event 2 
takes place when its probability exceeds some arbitrarily defined cut-off value, 
OL. Equation 8 is written as a rate with a MAX function to ensure that once an 
event occurs, E2 will remain equal to 1.0 for the rest of the run. In this manner 
any combination of events and impacts can be added to a system dynamics model 
using the DYNAMO computer language. 


Conclusion 


Discrete events do not constitute important forces in all systems. However, 
every modeling effort should at least include a consideration of whether or not 
events might be important influences on system behavior. In general, it is a fairly 
easy task to perform a preliminary test of the model's sensitivity to events. The 
analyst can simply use his own judgments about the probabilities and impacts of a 
set of events he feels might influence the system, add these judgments to the 
model, and test the sensitivity of the model behavior to the event impacts. If they 
do, indeed, cause important changes in model behavior, then a more extensive 
effort can be made to collect events and define event probabilities and impacts. 
Generally, the farther into the future the model is used to project, the more likely 
it becomes that the accumulation of event impacts will be important. 

Assessing the validity of a probabilistic system dynamics model is not greatly 
different from a model without events. In both cases the model must be tested 
under a variety of future conditions in order to assess its ability to respond 
reasonably to likely changes. It is impossible, of course, to test the cross-impact 
matrix by comparing its output with historical data simply because all of the 
events are future events. It is possible to add past events to the model to show 
how they influenced past behavior, and, in fact, this has been done,l as have 
historical validations of the cross-impact technique. 2>3 This does not, however, 
validate judgments about future event probabilities and relationships. Neverthe- 
less, even in a model without events there are usually relationships and judgments 
that do not affect historical calculations (such as areas of TABLE functions not 
reached by historical values of model variables) whose validity can only be assessed 
through a number of different runs to test the reasonableness of the model 
behavior. 

Finally, it is important to note that the arbitrary nature of the timing of 
event occurrences should be taken into account when the results of model runs are 
interpreted. The model may give a good representation of the probability of 
occurrence of each event. However, the timing of actual occurrences is highly 
arbitrary. In the real world, events may happen if their likelihood of occurrence is 
only .10 and those with likelihoods of .90 may not occur. No computational 
procedure used (whether it be based on random numbers or on a probability cut-off 
level or on some other system) will accurately predict when events will occur. 
However, by understanding this fact the analyst can perform several sensitivity 
runs and gain insight into the behavior of the system in response to event 
occurrences, whenever they happen, and how model behavior is likely to affect the 
timing of those occurrences. In this manner a more complete understanding of 
system behavior can be obtained than if future events had not been included in the 
model. 
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CROSS-IMPACT ANALYSIS 


The cross-impact analysis method is an approach by which the probabilities of 
an item in a forecasted set can be adjusted in view of judgments concerning 
potential interactions of the forecasted items. Most events and developments are 
in some way related to other events and developments. A single event, such as the 
production of power from the first atomic reactor, was made possible by a complex 
history of antecedent scientific, technological, political, and economic "happen- 
ings." In its turn, the production of energy from the first atomic reactor 
influenced many events and developments following it. In a sense, history is a 
focusing of many apparently diverse and unrelated occurrences that permit or 
cause singular events and developments. From these flow an ever-widening tree of 
downstream effects that interact with other events and developments. It is hard to 
imagine an event without a predecessor that made it more or less likely or that 
influenced its form— or to imagine an event that, after occurring, left no mark. 
This interrelationship between events and developments is called "cross-impact." 

The first step in a cross-impact analysts is to define the events to be included 
in the study. This first step can be crucial to the success of the exercise. Any 
influences not included in the event set, of course, will be completely excluded 
from the study. On the other hand, the inclusion of events that are not pertinent 
can complicate the analysis unnecessarily. Since the number of event pair 
interactions to be considered is equal to n^ - n (where n is the number of 
events), the number of interactions to be considered increases rapidly as the 
number of events increases. Most studies include between 10 and 40 events. 

An initial set of events is usually compiled by conducting a literature search 
and interviewing key experts in the fields being studied. This initial set is then 
refined by combining some closely related events, eliminating others, and refining 
the wording for others. 

Once the event set is determined the next step is to estimate the initial 
probability of each event. These probabilities indicate the likelihood that each 
event will occur by some future year. In the initial application of cross-impact and 
in some current applications, the probability of each event is specified, assuming 
that the other events have not occurred. Thus the probability of each event is 
judged in isolation and the result of the cross-impact analysis is to adjust those 
initial probabilities for the influences of the other events. 

The other approach to estimating initial probabilities assumes that the expert 
making the probability judgments has in mind a view of the future that serves as a 
background for his or her judgments. Thus in estimating the probability of each 
event, the probabilities of the other events are taken into account. In effect, the 
events are being cross-impacted in his or her mind. To ask that the effects of 
other events be disregarded would invalidate much of his or her expertise. In this 
approach the initial probabilities and impact judgments reflect the expert's view of 
the expected future situation. The cross-impact runs are used to show how changes 
in that situation (the introduction of new policies or actions, the unexpected 
occurrence of an event, etc.) would affect the entire set of events. 
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These initial probabilities may be estimated by individual experts but more 
commonly are estimated by groups containing experts from the various disciplines 
covered by the events. Delphi questionnaires or interviews also can be used to 
collect these judgments. 

The next step in the analysis is to estimate the conditional probability 
matrix. Impacts are estimated in response to the question, "If event m occurs, 
what is the new probability of event n?" Thus, if the probability of event n was 
originally judged to be 0.50, it might be judged that the probability of event n 
would be 0.75, if event m occurred. The entire cross-impact matrix is completed 
by asking this question for each combination of occurring event and impact event. 


When the initial probabilities have been estimated with reference to the 
other event probabilities (that is, not considering each event in isolation), there is 
some additional information that enters into the estimation of the impact matrix. 
For each event combination there are bounds on the conditional probabilities that 
can exist. These limits can be illustrated with a simple example. Suppose we 
considered two events, n and m, event n, with a 50 percent chance of occurring 
in the next year and event m, with a 60 percent chance of occurring. Thus, out of 
100 hypothetical futures, event n would occur in 50 of them and event m in 60. 
Obviously, events m and n would occur together in at least 10 of the futures. 


In this case, in answer to the questions, "If event m occurs what is the new 
probability of event n?" we are limited in our responses. A conditional probability 
of 0 for event n is impossible, for example, since if event n never occurred when 
event m occurred, the "overlap" of 10 combined occurrences would not be 
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percent of our hypothetical futures. Since, in this approach, it was assumed that 
the estimate of 0.50 for the original probability of event n included a 
consideration of the 0.60 probability of event m, an inconsistency in judgments has 
occurred. Either the original probability estimate of event n does not actually 
take into account the 0.60 probability of event m, or the probability of event n 
given the occurrence of event m is not equal to 0. One of these judgments is 
incorrect, because it leads to an inconsistency, but only the participants in the 
analysis can decide which judgment must be changed. They may decide that the 
initial probability estimate for event n did not fully account for the expected 
influence of event m, or they may decide that their original estimate of the 
probability of event n given the occurrence of m was too low. In either case, 
they have iearned something about events n and m because of the cross-impact 
exercise. This learning process that occurs while the cross-impact matrix is being 
estimated is one of the major benefits of performing a cross-impact analysis. 


The range of conditional probabilities that will satisfy this consistency 
requirement can be calculated easily. The initial probability of an event can be 
expressed as follows: 

P(l)= P(2) x PCl/2) + P(Dx P(l/2) (1) 


where: 

P(l) 
P(2) 
P( 1/2) 

P(2) 

P(l/2) 


probability that event 1 will occur 

probability that event 2 will occur 

probability of event 1 given the occurrence of event 2 

probability that event 2 will not occur 

probability of event 1 given the nonoccurrence of event 2 
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( 2 ) 


This expression can be rearranged to solve for P(l/2): 

rn/ ^ P(l) - P(2) x P(l/2) 

Pd/2) = nm 


Since P(l) and P(2) are already known (the initial probability estimates) and 
PC?) is simply 1 - P(2) only P(l/2) and P(1 /?), the conditional probabilities, are 
unknown. By substituting zero for P(1 ft) (the smallest value it could possibly 
have) the maximum value for P(l/2) can be calculated. Thus 

P(!/2) < w 


Similarly, by substituting 1.0 for P(1 ID (the largest possible value for P(l/5), the 
minimum value for P(l/2) can be calculated: 

P(l/2) < P(1) — (*> 


Thus the limits on the new probability of event 


are 


P(l) - 1 + P(2) 
Pl2] 


< P( 1/2) 



given the occurrence of event 2 


(5) 


Using equation (5) we can now calculate the limits for the example used previously. 
If the initial probability of event n is 0.50 and for event m is 0.60, the 
permissible values for the probability of event n given the occurrence of event m 
are 0.17 and 0.83. Or, if the probability of event n given the occurrence of event 
m is actually 1.0, then the initial probability of event n must be 0.60 or greater. 
Figure 1 presents the results of these limits calculations for any combination of 
probabilities for occurring and impacted events. 


Once the cross-impact matrix has been estimated a computer program is used 
to perform a calibration run of the matrix. A run of the matrix consists of 
randomly selecting an event for testing, comparing its probability with a random 
number to decide occurrence or nonoccurrence, and calculating the impacts on all 
of the other events due to the occurrence or nonoccurrence of the selected event. 
Impacts are normally calculated using odds ratios. To apply the odds ratio 
technique the initial and conditional probabilities of the events are converted to 
odds, using the following relationship 


Odds = 


Probability 
1 - Probability 


( 6 ) 


The impact of event n on event m is then calculated as the ratio of the 
odds of event m given event n to the initial odds of event m. Thus the cross- 
impact matrix shown in Figure 2 would become the matrix shown in Figure 3 when 
odds are used in place of probabilities. The ratio of the new odds to the initial odds 
is used to define the event impacts. Thus the occurrence of event 2 causes the 
likelihood of event 1 to go from odds of 0.33 to 1.50. The odds ratio expressing the 
occurrence impact of event 2 on event 1, therefore, is 1.50/0.33 = 4.5. Figure 4 
shows the entire odds ratio matrix corresponding to Figures 2 and 3. 
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Figure 1. Limits for Conditional Probabilities 
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Figure 2. Cross-Impact Probability Matrix 
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Figure 3. Cross-Impact Odds Matrix 
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Figure 4. Occurrence Odds Ratios 


A nonoccurrence odds ratio matrix also can be calculated from the 
information in the occurrence matrix in Figure 2. Again, using the equation 

P(l) = P(2)xPl/2) + PO)xP(l/2) 


the probability of event 1 given the nonoccurrence of event 2, P(l/5), can be 
determined. From these probabilities the nonoccurrence odds ratios can be 
calculated just as the occurrence odds ratios are calculated. 

Once the odds ratios have been determined the calculations proceed as 

foIlOWS! 


1. An event is selected at random from the event set. 


2. A random number between 0.0 and 1.0 is selected. If the 
random number is less than the probability of the event being 
tested, the event is said to occur. If the random number is 
greater than the event probability, the event does not occur. 

3. If the event (event j) occurs, the odds of the other events 
occurring are adjusted as follows: 


New odds 
of event i 


= (initial odds of event i) x (occurrence odds ratio 

of event j on event i) 


If the event does not occur, the same calculations are made 
but the nonoccurrence odds ratios are used. 
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4. Steps 1, 2, and 3 are repeated until all the events have 
been tested for occurrence. 

5. Steps 1 through 4 (which represent one play of the matrix) 
are repeated a large number of times. 

6. The frequency of occurrence of each event for all runs of 
the cross-impact matrix determines the new or calibration 
probability of that event. 

If the initial probabilities of each of the events included the impacts of the 
other events (the balanced matrix approach), the calibration probabilities resulting 
from the program run will normally be within a few percent of the initial 
probabilities, since the cross-impact matrix was constructed to be consistent (to 
include the consideration of the influences of the events on each other). However, 
since consistency is ensured only for event pairs, it is possible that inconsistencies 
in event triplets or quartets or higher orders may exist. The nonoccurrence impact 
matrix is calculated according to the assumption that the initial probabilities are 
consistent with each other. Thus, the nonoccurrence impacts should exactly 
balance the occurrence impacts. But these calculations are based on event pairs. 
The combined effects of two or more events on one event may cause some changes 
in the initial probabilities. If any of the calibration probabilities differ from the 
initial probabilities by more than a few percent, the matrix should be inspected for 
these higher order effects. 

If the initial event probabilities were estimated, assuming nonoccurrence of 
all other events, the calibration probabilities may be quite different from the 
initial probabilities. In this case the calibration probabilities are the new 
probabilities that result from the interactions of the events with each other. The 
cross-impact exercise has produced new estimates of event probabilities that take 
into account the interrelationships among the events. 

At this stage in the analysis the cross-impact matrix is ready to be used for 
sensitivity testing or policy analysis. Sensitivity testing consists of selecting a 
particular judgment (an initial probability estimate or a conditional probability 
estimate) about which uncertainty exists. This judgment is changed and the matrix 
is run again. If significant differences occur between this run and the calibration 
run, it is apparent that the judgment that was changed is an important one. It may 
be worthwhile to spend more effort in making that particular judgment. If no 
significant differences appear, that particular judgment probably is a relatively 
unimportant part of the analysis. 

Policy testing is accomplished by first defining an anticipated policy or 
action that would affect the events in the matrix. The matrix is then changed to 
reflect the immediate effects of the policy. This is accomplished usually either by 
changing the initial probabilities of one or more events or by adding a new event to 
the matrix. A new run of the matrix is then performed and compared with the 
calibration run. The differences are the effects of the policy. Often unexpected 
changes will result. When this happens these changes can be traced back through 
the matrix so that the chains of causality that lead to the unexpected changes can 
be determined and the effects of the policy understood. Used in this way the 
cross-impact matrix becomes a model of event interactions that is used to display 
the effects of complex chains of impacts caused by policy actions. 
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Performing a Cross-Impact Analysis 


Suppose that a study of the future of the chemical industry was in progress. 
In the course of the study, a list of important future events might be generated. 
One part of that list might include the following events: 

1. The use of plastics in transportation vehicles and 
construction expands sixfold from 1972. 

2. Increased governmental intervention in the process of 
innovation results from demands for consumer protection 
and pollution control. 

3. Chemical theory progresses to the point where much of 
chemical research can be done through computer calcula- 
tions rather than actual experimentation. 

4. The chemical industry expands into textiles through the 
development of nonwoven synthetic fabric. 

5. Chemical companies realize a declining return or rising 
investment in conventional research. 

The first step in using these events in a cross-impact analysis is to estimate 
initial probabilities for the events. Assuming that the balanced matrix approach is 
to be used, the probabilities might be estimated as follows: 



Event 

Probability of 
Occurring by 1985 

1. 

Use of plastics expands sixfold 

0.15 

2. 

Increased governmental intervention in innovation 

n r\ 

U.ZU 

3. 

Chemical research performed on computers 

0.25 

4. 

Chemical industry expands into textiles 

0.10 

5. 

Declining return on conventional research 

0.20 
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similar to Figure 5 is constructed. Each cell of the matrix represents the answer 
to the question, "If event x occurs, what is the new probability of event y?" For 
example, the first completed cell of the first row of the matrix contains the new 
probability of event 2 given the occurrence of event 1. Thus the question answered 
here is, "If the use of plastics in transportation and construction increases sixfold 
(event 1), what is the likelihood of increased governmental intervention in the 
innovation process resulting from the demand for consumer protection and pollu- 
tion control (event 2)?" Since the increased use of plastics is likely to increase 
demand for consumer protection and pollution control, event 2 should be somewhat 
more likely than initially estimated (0.20) if event 1 occurs. Thus we might judge 
that the new probability of event 2 becomes 0.30 if event 1 occurs. 


Since the influences of the events on each other were included in the initial 
probability estimates, this judgment must now be tested for consistency with the 
initial probabilities. By finding the intersection of row 3 (the 0.15 probability of 
event 1) and column 4 (the 0.20 probability of event 2) in the chart in Figure 5, we 
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If This Event Occurs: 


The Probability of 
This Event Becomes: 
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Figure 5. Sample Conditional Probability Matrix 

see that the limits on the conditional probability of event 2 given event 1 are 0.0 
and 1.00. Thus, no problem is presented by the judgment of 0.30 for the probability 
of event 2 given event 1. 

In a similar fashion the entire matrix is completed. The next task is 
specifying policy or sensitivity tests to be run with the matrix. In this case, for 
example, we may wish to know the effect on the other events if event 3 (use of 
computers for much chemical research) occurs. Thus one test would be performed 
by assigning a probability of 1.0 to event 3 and rerunning the matrix. A second test 
might be performed to test the sensitivity of the events to event 2 (increased 
governmental intervention in the innovation process). These tests are shown below. 


TEST OF OCCURRENCE OF EVENT THREE 
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TEST OF SENSITIVITY TO EVENT TU O 
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APPENDIX E - CONSENSOR 

A DESCRIPTION OF THE CONSENSOR™ 

The CONSENSOR is a decisionmaking tool that functions as an aid in the 
determination of group consensus. It facilitates anonymous, weighted voting and 
automatically averages a group's many shades of opinion while taking into account 
the feelings of confidence each person has in his own opinion. It enables each 
participant to express not only subtle changes in personal opinion, but also the 
various levels of confidence or conviction that support those opinions and even his 
own self-assessed qualification to vote on any given subject. In addition, this 
system enables each individual to participate fully with a highly flexible degree of 
anonymity. 

The CONSENSOR enables all participants in a discussion to say how they feel 
about the subject at hand in two dimensions: (1) each expresses agreement or 

disagreement with a given problem statement by means of an electronic switch 
which is graduated through a range of eleven possibilities, identified by the 
numbers 0 through 10 (0 is usually assigned the lowest, least important, or least 
probable value, while 10 is the highest, most important, or most likely value) and 
(2) each participant selects one of the five "weighting" values on a second switch; 
these are graduated in 25 percent increments from 0 through 100 percent and 
actually qualify (give weight to) the previous selection according to the amount of 
confidence the participant feels or wishes to express. 

Thus, "weighting" qualifies any opinion expressed by the "selection" control. 
The value or the quantity to which each weighting switch is set determines the 
intensity of that participant's vote or selection. A participant who sets the 
weighting switch at 50 percent registers an opinion (selection) at half strength. 
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Regardless of the vote or selection input by the selection switch, it is counted as a 
full vote if the weighting switch is on 100 percent, but only as half a vote if the 
weighting switch is at 50 percent. Therefore, if half of a given decisionmaking 
body votes with 100 percent intensity, and the other half of the group votes with 
50 percent intensity, the average weighted input from the group as it appears on 
the display screen will be 75 percent. 

When voting on a series of alternatives or a range of selections, each is 
assigned one of the 11 values around the selection switch from 0 through 10. 
Selections are registered anonymously, and the group's aggregate "weighted" 
position is displayed in a large histogram form, visible to all. Cumulative weighted 
input to each of the 11 selections is displayed as a distribution that is always 
normalized to 100 percent. Group consensus, when it occurs, appears on the 
display screen in the form of a bell-shaped curve generated by columns of light. 

Relative degrees of consensus are shown by the distribution of opinions — the 
taller a normally distributed curve, the more harmonious the group's thinking. A 
bimodal curve indicates disparity of opinion (usually due to an ambiguous problem 
statement), while a low, wide curve shows diversity of opinion. In addition, the 
display screen shows the weighted mean (average) opinion within the distribution 
and the average weight (intensity) expressed by the group as a whole. 

Highly flexible in its applications, the CONSENSOR is particularly useful to 
committees, review boards, and task groups. Its applications range from assess- 
ment of alternatives, plans and strategies, to advertisement pretesting, personnel 
evaluation, and simple "make your choice" voting situations. It forces clear 
problem definition, which itself produces shorter and more effective meetings. It 
encourages full participation and open discussion, thus providing a register of the 
true feelings of key people. 
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The value of this system to communication lies in its direct depiction of 
opinions on a visual scale. The display thus pictorially reveals areas of misunder- 
standing before time is wasted on misdirected discussion or on clarification 
measures that would be tedious to anyone present who might not require them. 

The CONSENSOR Model 3000 offers the following major innovations in design 
and functions 


1. A microprocessor technology that reduces the component count, 
thereby greatly increasing its reliability. In addition, a micropro- 
cessor provides the CONSENSOR with greatly expanded calculation 
capabilities. 


2. Cathode ray display of all information within one digit accuracy. 
The full distribution of input to all 11 selection positions from each 
of the participants, as well as the average group weight and the 
weighted mean, is shown correct to the nearest decimal. In 
addition, the CONSENSOR Model 3000 calculates and displays by 
CRT previous iterations, stored in memory. 


3. 


The unit is easily portable. The display is no heavier than a standard 

XV unit. Thp \mtor +^r r P tna Ic ora K o r rliwir in , m 1 A -f C 
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designed for simple installation by the users. The entire installation 


can be made on a conference room table or around the edges of both 
"U-" and "H-" shaped tables. 


4. Permanent installations can be made by securing the wire connec- 
tions under the table top where they are out of sight and out of the 
way when the CONSENSOR is not being used. The individual voter’s 
terminals are permanently wired in modular units of 5. Each 
CONSENSOR system is delivered with 3 modules, each simply 
connected to the others in any sequence by the user, thus comprising 
a basic unit for 15 participants. 

5. Design of a voter terminal unit that assures the anonymity of each 
participant (to the degree that each person desires it). Each "vote" 
is input by pressing a "vote activate" button, while both selection 
and weight settings are screened from view on each side. 


Options are available for users who wish to expand the applications of the 

CONSENSOR or to extend the usefulness of data generated by any given meeting. 

Each option would carry a moderate additional cost. 

1. Additional Terminals . Additional voter terminals can be provided in 
modular groups of 5, all of which are easily added to the basic set of 
15. Any number of additional units can be "stacked," up to a total 
of 60. 


89 


2. Math Package. The Model 3000 can be programmed to display (or 
record on hard copy) any additional statistical data required by the 
users. Such information as median, mode, standard deviation, 
equivalent unweighted means, and skewness can be made instantly 
available to the group on a continuing basis, or only as needed. 
Control of displaying this information is by means of an optional 
"activate" switch on the moderator's panel. 

3. ComDuter Connection. The system can be interconnected, by way 
of an RS-232 connection system, with a computer, permitting, in 
addition to increased storage capability, computer analysis of data 
generated during any given meeting for display on the CONSENSOR 
screen. This permits interaction of data from the group and from 
outside data banks. 

k. Hard Copy Printout . Results are printed alphanumerically on a 
paper tape. Each CONSENSOR iteration is identified by a serial 
number so that a record can be kept of each series. The printer is 
designed with an "active" control and thus functions as a discre- 
tionary option throughout a meeting. 

5. Cost Clock . A digital display of the running cost of a meeting, 
calibrated to show dollars rather than minutes, and adjustable based 
upon the average annual salary of participants. 

Thus it can be seen that the CONSENSOR is a new computer tool to make it 
easier for planners and decisionmakers to discover how much they know and how 
strongly they feel about alternatives facing them. The CONSENSOR offers a way 
to make business meetings and public discussions shorter, more productive, and at 
the same time more democratic and representative of the participants' true 
beliefs. 
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APPENDIX F - SCENARIOS 


SCENARIOS: THEIR CONSTRUCTION AND USE 
What Is a Scenario ? 

A scenario is a policy analysis tool. It describes a possible set of future 
conditions. In general, the term scenario has been used in two different ways to 
describe future conditions: first, to describe a snapshot in time or conditions of 

important variables at some particular instant in the future; second, to describe a 
future history, that is, the scenario can describe the evolution from present 
conditions to a given point in time. The latter approach is generally preferred 
because it can provide causal information regarding developments within a given 
scenario. The most meaningful and useful scenarios are those that display 
conditions of important variables over time covering the period of interest. In this 
latter approach the evolution of conditions, or of the variables, is described by 
narratives that treat the important events and developments that shape the 
variables. 

When scenarios are used in policy analysis, the nature of evolutionary paths is 
often important— since policies can act to deflect those paths. In policy studies 
families of scenarios often are used to illustrate the consequences of different 
initial assumptions, different evolutionary conditions, or both. For example, a 
study of transportation policy might involve constructing several scenarios that 
differ in their assumptions about birth rates, population, migration, economic 
conditions, and costs and availability of various forms of energy. When a set of 
scenarios is prepared, each scenario usually treats the same or similar parameters, 
but the evolution and actual value of the parameters described in each scenario are 
different. 
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Clearly, a set of scenarios can describe a range of alternative futures that 
differ only as a result of the initial conditions and/or the flow of events that shape 
the conditions described in the scenario. Sets of scenarios may be generated by 
changing one or several key assumptions. For example, in the above-cited case, 
birth rate might be assumed low in one scenario, moderate in another, and high in a 
third. In addition, it might be important to reach an understanding of conditions 
that result not only from demographic variations (e.g., birth rate) but also from 
other conditions such as the development and use of alternative energy systems. 
The dimensions usually are chosen to represent the leading or most influential 
variables (in the above example, birth rate, and energy costs and availability) from 
which other conditions will evolve. 

In preparing scenarios, an analyst has to make several crucial choices: 

Specifically, what variables should be included in the scenario 
(e.g., those that are most important to shaping transportation 
demands)? 

If more than one scenario is to be used, in what way should the 
scenarios in the family differ? How large should the difference 
among scenarios be to make them distinguishable from one another 
in order to allow meaningful policy analysis? 

What conditions or issues should be studied? 

How can internal consistency be assured within each scenario 
(e.g., how can mutually exclusive events and trends be excluded)? 

This is an important question, since each scenario must provide a 
rational explanation of the sequence of events described. 

Each scenario usually has several major components or dimensions, 
e.g., demography, the economy, natural resources, energy, social and institutional, 
technology, etc. 

How Is a Scenario Constructed ? 

The process of scenario building begins by defining a candidate list of 
demographic, socioeconomic, and institutional variables that are important to the 
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subject being studied. (See Table 1 for a typical list used in recent studies.) 
Second, a list or set of characteristics of a particular system (transportation, 
energy, etc.) is determined. The first list is often compared with the second to 
assure that important external variables (e.g., to the transportation system) are 
included. The events that are described in a scenario, and that help shape 
projections of the variables, are critical. An event is best described as a discrete 
occurrence that has some finite chance of happening in the future time period 
under study. Clearly the likelihood of occurrence of certain events will— and 
should— differ in each scenario and, in fact, these events help discriminate between 
the scenarios. (See Table 2 for key events typical of those used in these studies.) 

A scenario, or set of scenarios, composed only of extrapolation of past trends 
surely would be an unlikely descriptor of possible future conditions. In the real 
world, the occurrence of "unprecedented" events continually interrupts trends in 
progress. The chronological history of any system or subsystem is partially written 
in terms of unprecedented events (railroad development, the automobile, the OPEC 
oil embargo). They often are the essence of an era. When scenarios are 
constructed, it is important to search for events that are plausible and relevant to 

the subject at hand. Here the question focuses on: What important events 

technological, demographic, economic, societal, etc — logically might be expected 
to occur within the time frame of interest? Sometimes interesting and instructive 
scenarios can be constructed by including low-probabiltiy, high-impact events. 
When low-probability, high-impact events are introduced, they often can become 
the focus of the scenario and can determine its outcome and the evolution of the 
variables that are described in the scenario(s). Such events can be used in "what- 
if" situations, e.g., What if nondepletable energy sources are discovered? 

Yet there are no analytic methods that yield events for a scenario. The 
procedures employed to generate events can stimulate, collate, organize, extract, 
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Table 1 


PROJECTED INDICATORS 


Total U.S. Population (including armed forces abroad) 

- U.S. Population Ages 5-19 

- Resident Population in the Combined South and West Census 
Regions as a Percentage of Total Resident Population of the U.S. 

- Population Living in Urban Areas (using the 1970 definition of urban 
areas) as a Percentage of the Total Resident Population in the 
Combined South and West Central Regions 

Percentage of U.S. Land in Urbanized Areas 

Number of Households 

Total Labor Force 

- Total Labor Force Participation Rate (number of persons 16 years of 
age and employed or actively seeking work as a percentage of the 
total noninstitutional population 16 years of age and over) 

- Gross National Product (constant 1975 dollars) 

- Gross National Product Per Capita (constant 1975 dollars) 

- Disposable Personal Income Per Capita (constant 1975 dollars) 

- Total Government Expenditures (federal, state, and local) as a 
Percentage of GNP 

- Total Federal Expenditures (constant 1975 dollars) 

AAA Bond Yield 

Final Sales of Goods as a Percentage of Total Final Sales 
Labor Productivity 

- Nonresidential Fixed Investment 
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Table 2 


IMPORTANT EVENTS USED IN SCENARIOS 


Government subsidizes relocation and training of needy rural workers to 
encourage migration to urban centers as a component of federal social 
welfare programs. 

Five new cities of 50,000 population are developed proximate to natural 
resources (i.e., coal) to facilitate new resource development. 

Increased capital demand puts upward pressure on interest rates. 


- Selective wage, price, profit, and interest rate controls are periodically 
imposed to guide production, consumption, and investment patterns. 

- Middle-class attitudes toward work are eroded by the rise of strong 
avocational interests, resulting in decreased demands for career 
advancement opportunities. 


Nearly all workers undergo job retraining because of technological 
obsolescence or voluntary career change with federal and state govern- 
ment support supplementing private industrial programs. 


R<5cD spending in the United States increases from the mid-1970 level of 
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Accelerated depreciation allowances are approved and become law 
(20 percent increase over 1975 levels). 


Dislocations caused by governmental policies result in a cyclical 
downturn where capacity utilization in manufacturing faiis to 
70 percent and remains there for eight consecutive quarters. 

The stock of capital per worker averages 2.5 percent growth for a 
ten-year period. 


Legislation is passed providing a guaranteed minimum annual liiCOme 
for U.S. citizens. 


Legislation is passed that imposes effluent fees on industrial processes 
that are environmentally degrading; the fees collected are allocated to 
environmental R&D. 


Federal funds for community development, to revitalize cities, increase 
threefold over the 1975 level. (Community development funds totaled 
$3.2 billion in 1975.) 

Successful legislation is enacted guaranteeing full employment by 
specifying the federal government as the employer of last resort. 

European Community and 3apan erect prohibitive trade and investment 
restrictions that restrict market access by the United States. 
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and file possible occurrences. They cannot, however, be expected to bring to the 
surface concepts that have not yet been articulated. 

The projected values of variables and the general conditions of the scenarios 
must be examined since scenarios usually are developed from a large number of 
trends. The projections of the variables are examined to see whether they contain 
contrary implications about socioeconomic, demographic, institutional, and other 
behavior. The projected variables also are scrutinized for "reasonableness"~i.e., in 
terms of whatever knowledge there is about the growth process of the particular 
variables being discussed. (See Figure 1 for two trends projected in this type of 
work.) This process usually starts at the macro level and filters down to the details 
of the particular system for which the scenario was constructed. 

Why Are Scenarios Used ? 

As noted earlier, scenarios provide a context for policy analysis. To be really 
meaningful, a set of scenarios should spotlight policy intervention points where 
critical roles are likely to be found. Certain actions, while producing some 
favorable public and private sector results, may produce others that are 
undesirable. Hence, the scenario set should provide material to spotlight policies 
and actions that are likely to maximize desired benefits. 

In this regard, one of the most important factors in evaluating alternative 
policies and actions will be the ability, from the scenario work, to assess political 
and administrative feasibility. Such needs might include regulatory changes, new 
statutory authority, and even government reorganization. Also, budgetary costs, 
manpower, and other resources can be important considerations in policy analysis. 
Political and administrative feasibility of possible alternative actions cannot 
realistically be ignored in policy analysis. 
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Figure 1. Two Projections Used in Scenarios 



In summary, a scenario is not a forecast per se. Rather, it is a plausible 
description of what might occur. This form of presentation describes events and 
trends as they could evolve. The likelihood that any particular scenario actually 
will be realized is quite low for several reasons. There are a number of events and 
trends that are discussed explicitly in each scenario. Since the likelihood of 
occurrence of the bulk of these is less than unity, their product (that is, the overall 
probability of the scenario) must be small. 

Furthermore, scenarios are necessarily incomplete for two basic reasons. In 
the interest of brevity, scenarios usually focus only on those aspects of the future 
believed to be crucial to the central problem being addressed. Also some future 
developments always will be unforeseeable; that is, they are inaccessible by any 
forecasting technique. Nevertheless, scenarios have proved to be an effective aid 
for preparing and testing policies, even though they do not "depict the future." 
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APPENDIX G - TREND IMPACT ANALYSIS 


TREND IMPACT ANALYSIS 


Trend Extrapolation 

The ability to quantify various parameters and project them into the 
future is an important factor in accomplishing the various steps in the 
planning process. Many techniques are and have been used to obtain such 
time-series data. These range from highly judgmental, intuitive methods 
to highly complex mathematical treatments. In the former case, individual 
estimates (genius, expert, or nonexpert) or group consensus (obtained by 
polling, face-to-face conferences, Delphi conferences, or situation gaming) 
may be employed. 

Unfortunately, both human judgment and mathematical extrapolation 
have their own fallibilities. Past combinations of the two have not been 
notably successful in combining the best features of each, while avoiding 
their weaknesses. A principal strength of judgment in trend extrapolation 
is that humans can take into account the possible impacts of unprecedented 
future events that may cause unique perturbations in the trends — for example, 
if a pharmaceutical industrialist or food manufacturer might be interested 
in how the discovery of a link between pancreatic cancer and the consumption 
of sugar would influence the trend in sales of artificial sweeteners. This 
influence, however, could manifest itself in quite novel ways, since, by 
its very nature, it has never been felt before. Common mathematical methods 
of extrapolation are unable to take into account potential future events 
since the past history of a trend cannot reveal how it would be influenced 
by such events. 
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On the other hand, subjective or unaided human judgment usually is 
inferior to mathematical formulas when it comes to fitting a set of points 
with a best-fit curve. Moreover, mathematical curve-fitting techniques 
are well established in theory and application. They can be readily 
communicated and used by others, thereby overcoming the stigma of arbi- 
trariness or mere idiosyncrasy that attends subjective projections. At 
best, however, a mathematical approach simply produces a good estimate of 
how a trend would appear if it were not modified by unprecedented future 
events . 

Three examples illustrate how the occurrence of unprecedented events 
can influence a previously stable trend. 

1. Until about 1955 the fertility rate in the United States 
rose regularly and smoothly. The trend reversed 
dramatically in the 1960s when cheap and effective 
contraceptives permitted the expression of new values 
and attitudes about ideal family size (see Figure 1). 
Extrapolations based on the historical trend from 1950 
through 1960 consistently overestimated the present 
birth rate in the United States. 

2. Figure 2 illustrates the long-term drop in the cost 

of electricity in the United States. The trend toward 
diminishing costs began almost with the advent of the 
first electricity generating system and reflected the 
generally unstated goal of producing cheap power. Cost 
was reduced through economies of scale, improved tech- 
nology and operating efficiency, more readily available 
fuels, etc. Recently, however, the cost of electricity 
has stabilized and begun to rise because of increasing 
costs of fuel, new requirements for costly antipollution 
devices, and restrictions on the size of generating 
plants that end or lower savings through economies of 
scale. An extrapolation based on all but the latest 
data would have missed the recent "turn around." 
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D.2 FERTILITY RATE* 


NUMBER PER 1000 FEMALES AGE 15-44 



Source of historical data: Public Health Service, National Center for 

Health Statistics, Vital Statistics of the United States, 1963 . Vol. 1, 
Monthly Vital Statistics Reports . Vol. 19, No. 13; Vol. 20, No. 13; and 
Vol. 21, No. 12, as cited in Executive Office of the President, Office of 
Management and Budget, Social Indicators 1973 . Table 8/5, p. 252; U.S. 
Bureau of the Census, Statistical Ab stract of the U nited States 1974 
(Washington, DC: U.S. Government Printing Office, 1974), Table No. 9, 

p. 11. 


♦General fertility rate is the number of births per 1000 resident females 
15-44 years of age. 


Figure 1. Fertility Rate in the United States 
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Cents 


4.0 



Year 

SOURCE: Edison Electric Institute, Questions and Answers 

about the Electric Utility Industry , annual. 

Figure 2. Electricity Used in the Home (average cost per 
kilowatt-hour) 
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3. The long-term trend in the United States toward the 
sale of automobiles of increasingly greater weight 
and horsepower has begun to change appreciably in 
recent months. This change may be the result of 
concern about increasing costs of gasoline, new 
public attitudes about conservation of the environ- 
ment, or both. 

Many other examples can be cited, of course. The point is that 
deviations from historically based extrapolations usually seem to reflect 
the impact of unprecedented events. 

Trend impact analysis (TIA) , an analytic procedure developed by 
The Futures Group, divides the task of extrapolation in such a way that 
humans and computers are assigned precisely the task that each does best. 
First, the computer extrapolates the past history of a trend. Second, 
the human specifies a set of unique future events and how the extrapola- 
tion would be changed by the occurrence of each of these events. The 
computer then uses these judgments to modify the trend extrapolation. 
Finally, the human evaluates the resultant adjusted extrapolation and 
modifies the input data to the computer in those cases where the output 
appears unreasonable. Figure 3 shows this procedure schematically. 

Mathematical Trend Extrapolation 

The development of a surprise-free extrapolation is the first step 
in the TIA process. A computer program selects the "best fitting" curve 
from a set of alternative equations. This curve then is used to provide 
the surprise-free future extrapolation. At the option of the program 
user, in order to avoid unreasonable extrapolations, the program can 

either truncate extrapolations that fall outside upper or lower bounds, or 
select the "best fitting" curve only from among those that do not give rise 
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JUDGMENT 



Figure 3. Trend Impact Analysis (TIA) 




to extrapolations falling outside the specified bounds. Alternatively, 
the user can reject the mathematical extrapolation generated by the TIA 
program and supply an extrapolation developed by some other curve-fitting 
program or one based entirely on human judgment. 


Several refinements in the programming of this aspect of TIA enhance 
the effectiveness of the best-fit test and extrapolation procedure. 


1. It is not necessary that the data cover a continuous span 

of time. Data in which there are gaps are fully acceptable — 
the program makes use of whatever data are available, taking 
into account any gaps, but without being stymied by them. 

2. The program does not give equal weight to all data. Rather, 
a year may be specified (normally the present year) for which 
data are to be given maximum weight. As the times to which 
the data refer are further removed from the year that has 
maximum weight, the data are given less weight.* This pro- 
cedure thus takes into account the possibly lower reliability 
of data that are more distant in the past or, more important 
perhaps, the lower influence on the future of developments 
that have occurred progressively farther in the past. The 
formula chosen also makes the sum of an infinite number of 
weights infinite, rather than convergent, so that even very 
distant years continue to have a finite contribution. 


3. Since there 
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reports to the human user just how good the fit was, using 
the same squared correlation coefficient that determined 
which mathematical formula should be used. As noted earlier, 
where judgment or analysis indicates a more realistic set 
of data should be used, they can be input directly as part 
of the specified data used for subsequent steps. 


4. Upper and lower limits on the extrapolation may be set. In 
this case any curve that produces an extrapolation that ex- 
ceeds these limits will be rejected. Thus the extrapolation 
is based on the best-fitting curve that does not exceed the 
specified limits. 


*The weighting formula is 1 . where y 

1+ |y-y 0 

is the year given maximum weight. 


is a given year and y^ 
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Human Judgments of Event Impacts 


Human judgment and imagination are central to the second step of 
TIA. Here, the program modifies the surprise-free extrapolation to take 
into account important, unprecedented future events. First a list of 
such events is prepared. These events should be unprecedented, plausible, 
potentially powerful in impact, and verifiable in retrospect. The source 
of this list of events might be, typically, a literature search, a Delphi 
study, or a consensus among consultants. Whatever the source, the events 
selected comprise an inventory of potential forces that could lead to a 
departure from a surprise-free future. 

Several judgments are made about each selected event. First, estimates 
are made of the probability of occurrence of each event as a function of 
time. Second, the impact of each event on the trend under study is esti- 
mated. Impacts can be specified in several ways; our procedure (Figure A) 
involves specification of the following: 

1. The time from the occurrence of the impacting event 
until the trend begins to respond. 

2. The time from the occurrence of the impacting event 
until the impact on the trend is largest. 

3. The magnitude of that largest impact. 

A. The time from the occurrence of the impacting event 

until the impact reaches a final or steady-state level. 

5. The magnitude of that steady-state impact. 

Each of the three specified times and the impact magnitudes associated with 
them are taken to be completely independent. For example, the maximum 
impact might be positive, and the steady-state impact negative, or the 
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TIME TO STEADY-STATE OR CONSTANT IMPACT 


steady-state impact might be zero, meaning that the impact is only temporary. 
Finally, the maximum impact might be the same as the steady-state impact. 

In addition, impacts can be specified in either relative or absolute 
units — i.e., they can be specified as percentages of the values of the 
trends at the time of impact, as a percentage change of that number, or 
they can be specified in absolute units of magnitude of the trend. For 
example, the impact of a particular event on the number of dentists could 
be specified either as 90 percent of that number, as a 10-percent decline 
of that number, or as a downward shift of 12,000. The form used to record 
these estimates is shown in Figure 5. These impacts are calculated, when 
sufficient information is available to do so. Otherwise they are judg- 
mentally determined. 

Computer Processing of Impact on Extrapolated Trends 

The heart of TIA is the computer program that uses these judgments 
to calculate the expected impact of the selected events on the extrapo- 
lated trend. A closed-form procedure is used to solve this problem. The 
expected value, or mean, of the impact and upper and lower quartiles of 
the distribution of possible impacts are computed for each indicator. The 
expected value of the impact is computed by summing the products of the 
probabilities of the impacting events for each possible year times the 
magnitude of their impact, taking into account their specified time lags. 
Probabilities of events for years not specified are estimated by linear 
interpolation, assuming that an event has 0.00 probability at the present 
time. Similarly, impacts are linearly interpolated between the three 
specified impact magnitudes. 
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This approach treats the coupling among the impacts of the various 
events as negligible. Thus, the impact estimate is produced as the sum 
of independent random variables. The net result is that the variance of 
the impact-adjusted forecast is the sum of the variance of the trend 
extrapolation (as measured by the square of the standard error of esti- 
mate) and the variances of the impacts of the associated events. 

Thus, where P is the likelihood that event e will occur in year y, 
’ ye — 

and ay k ~y,e is the impact that event e would give rise to (y^-y) years 
after its occurrence, the expected value of the impact in year y^ would be 


5 2 

e y=y o 

Figure 6.) 
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where 


is the present year (e.g., 1975). 
o 


(See 


Typical TIA Results 

Use of the TIA procedure has revealed that important insights may be 
obtained by utilizing this form of trend extrapolation. The development 
of improved trend forecasts is only one of the advantages of this method. 
Insight into how adjustments of event probabilities and impacts vary the 
estimated future value of the indicator in question, in terms of both the 
median and interquartile range, also can prove to be very useful in devel- 
oping an understanding of the effectiveness of policies or actions that 
may be available to us. 

The forecast of the indicator shown in Figure 7, the average cost of a 
prescription, is drawn from a recent report that is part of a data service 
(called PROSPECTS®) developed at The Futures Group. The forecasts in the 
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Figure 6. Expected Value of an Event Impact 
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AVERAGE COST OF A PRESCRIPTION 


(Constant 1970 Dollars) 



SOURCE: R. A. Gosselin & Company, IMS America 

Figure 7. A Typical Forecast Obtained Using 
Trend Impact Analysis (TIA) 
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PROSPECTS reports are prepared using the TIA procedure and, as they represent 
material prepared to aid in real-world decisionmaking and planning, should 
prove useful in discussing the insights obtained from using TIA. 


Initial or Baseline Extrapolation 

It should be remembered that the impacts assigned to each event describe 
the estimated change in the surprise-free trend caused by the occurrence of 
that event. In the case of the average cost of a prescription, an upper limit 
of $8 per prescription (in 1970 dollars) was set for the extrapolation. The 
extrapolation program rejected the first three curves generated because they 
exceeded this limit. The fourth curve remained within the limit and produced 
the solid-line extrapolation shown in Figure 7. This, then, became the base- 
line to be impacted by future events. 


Event Impacts 
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first event, the abolition of all product brand names, was judged to have 
a probability of .10 of occurring by 1985 and a probability of .15 of occur- 

~ ~ 1*,, loon T-P noonr ir i c that *? t c first irtmart 

1- J- W u y v » 4- A W w.ww*. — — — — — r * 

on the average cost of a prescription will begin two years after the occur- 
rence of the event. The maximum impact will occur after five years and will 
be a 20-percent reduction in the average price. The steady-state impact 
is judged to be the same as the maximum impact. 

The combination of these events, probabilities, and impacts with the 
baseline extrapolation is a forecast (Figure 7) markedly different from 
the baseline extrapolation. The curve even begins to decline in 1987. 


113 
















The uncertainty is indicated by quartiles about 18 percent above and below 
the mean forecast. (The quartiles indicated the middle 50 percent of future 
values of the curve. Thus, 25 percent of the futures lie above the upper 
quartile, 25 percent lie below the lower quartile, and 50 percent lie 
between the two quartiles. Quartiles are presented here; however, since 
the computer program calculates the standard deviation, skewness, and kurtosis 
for each year, any part of the range could be printed out.) This uncertainty 
shown by these quartiles results from the fact that many of the events that 
have large impacts have relatively low probabilities — thus an uncertain 
situation prevails. 

At this juncture, it is desirable to determine the sensitivity of these 
results to the individual estimates upon which they are based. For example, 
one might raise valid questions about the estimates of event probability, 
the magnitude of the impacts used, and the delay time associated with these 
impacts. Having prepared these data in a disaggregated fashion, it is 
extremely easy to vary such estimates and view the change in results. It 
also may be observed that intervention policies, whether they be institutional 
(such as lobbying, advertising, or new marketing approaches) or technological 
(such as increased R&D expenditures) , can be viewed as a means of influencing 
the event probabilities or impacts. 

Suppose, for example, a certain pharmaceutical company were in a position 
to lobby for the immediate removal of restrictions on prescription advertis- 
ing, or suppose an analyst thought that the removal of these restrictions was 
much more likely than 20 percent in 1980. In each case, knowledge of the sensi- 
tivity of the forecast to the removal of advertising restrictions would be 
useful. This sensitivity can be tested by raising the probability of this 
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event from .20 in 1980 to .90 in 1980. The result of this change is shown 
in Figure 9. 

Figure 9 shows the sensitivity of the forecast to an early occurrence 
of this event is mainly during the 1975-1985 period. During this period 
the forecast is reduced by about 7 percent and the quartiles are similarly 
reduced. By 1990, however, when the probability of the event had already 
reached .60 in the first forecast, the difference is slight. The sensi- 
tivity of the forecast to each of the other events, or combinations of 
events, can be determined in a similar manner. 

Thus TIA can be used, not only to improve forecasts of time-series 
variables but also to study the sensitivity of those forecasts to policy. 

Of course, any policy considered should attempt to influence as many events 
as possible rather than one, as in this simple example. Realistically, 
corporate actions often have both beneficial and detrimental possibilities, 
as they may enhance both desirable and undesirable possibilities. The use 
of such procedures as described here, however, should make such uncertain- 
ties more clearly visible than is possible with techniques heretofore 
available and allow us to live more comfortably with, and even to reduce, 
the degree of risk in our endeavors. 
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APPENDIX H - STATE-OF-THE-ART ANALYSIS 

STATE OF THE ART ASSESSMENT TECHNIQUE 

This chapter describes our method for measuring and forecasting technology 
state of the art and how this method was utilized for cross-national comparisons of 
technology, (u) 

The State-of-the-Art Measurement Techniques (u) 

Definition of State of the Art . Although "state of the art" is a familiar term, 
it lacks precision. Generally, when a technology is described as state of the art, it 
is taken as an example of an advanced development in the field, but there is no way 
of indicating the degree of advancement. The Futures Group has developed a 
convention for measuring technology state of the art utilizing an index comprised 
of selected performance parameters that describe a particular technology. The 
approach has proved versatile in its ability to capture technological performance at 
various levels of system aggregation and in relating increases in state of the art to 
developments in component technologies and advances in design. By plotting the 
state-of-the-art indicator for each new product/innovation over time, the path of 
technological development can be quantitatively described, (u) 

In this convention, the state of the art (SOA) of a particular product or 
process is defined as a linear combination of a number of factors or parameters 
descriptive of that product or process. While nonlinear equation forms can be used, 
the basic functional form of the state of the art is as follows: (u) 

SOA = K i(Pi/P'i) + K2(P2/P’2) * * * K n (P n /P’ n ) 0) 


118 




where n is the number of parameters that are used to define the technology, P n is 
the value of the n*h parameters, P' n is a reference value of the n^ parameter (used 
to nondimensionalize the equation), and K n is the weight— that is, the relative 
importance of the n*h parameter, (u) 

This equation defines progress in the technology as improvements in the 
parameters selected to describe the technology. The relative contribution of each 
of the parameters to the overall state of the art is determined by the selected 
weights. Selection of the parameters which describe the technology and the 
weights with which these parameters are applied are key issues, and both 
judgmental and statistical methods have been developed for parameter and weight 
selection, (u) 

To further explain the method, imagine its application to computers. Suppose 
we are interested in determining the state of the art of mainframe computers, used 
in a variety of scientific and commercial applications. We would begin by listing 
all mainframe computers for which data are available and their dates of introduc- 
tion. The parameters used to describe these computers might include, for example: 
cost, memory size and speed of operation. In short, the parameters in the 
aggregate answer the question: "What is this technology designed to achieve?" An 
excellent technology— that is, one that is high with respect to its state of the art— 
would be one that achieves its designed ends better than a lower state-of-the-art 
example. This implies that a mainframe computer that rates very high in its 
intended application may well rate very low when it is evaluated in another 
application. Technological progress is defined by a succession of products or 
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processes that continually improve in their ability to achieve the application for 
which the technology or process was designed, (u) 

A measure of the sort proposed here has many uses: (u) 

- A newly introduced product or process can be compared with other 
extant products or processes designed to perform the same mission; 
thus this approach can be used as a means for tracking progress in a 
particular field. The advancing state of the art of a series of 
machines designed to accomplish a particular purpose can be fol- 
lowed over time. 

Technologies produced in different countries can be compared to 
determine which country has the lead and the extent of the lead 
time. 

By having a time history of the state of the art of a given field, 
forecasting can be enhanced; an extrapolation of the historical 
state-of-the-art curve can lead to rational expectations about 
improvements in performance over a future time interval. 

The significance of "breakthroughs" can be measured by observing 
how a given improvement in a product is manifested in terms of a 
"jump" in the state-of-the-art curve. 

- State-of-the-art indices for an industry or even an entire economy 
might be constructed by aggregating the state-of-the-art curves for 
key technologies within the industry or economy. 

- Finally, the use of state-of-the-art convention enhances communi- 
cation among technologists and between technologists and users of 
their products and processes since the state-of-the-art method 
requires that application and parameter weight assumptions be made 
explicit, (u) 

Research at The Futures Group, sponsored by the National Science Founda- 
tion and others, has permitted us to investigate the application of the linear form 
of the state-of-the-art equation to a number of technologies including: computers, 
antibiotics, farm tractors, high-temperature materials for turbine blades and 
vanes, turbojet engines for transport aircraft, man-made fibers, and coal gasifica- 
tion. This prior research has also permitted us to examine and compare several 
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methods for collecting expert judgment about the specific parameters to be used as 
well as their weights, and to develop statistical techniques for computing the 
weights from historical data. It is worth reiterating that we were not attempting 
to construct a finely tuned and elegant theoretical model based on fundamental 


natural laws; rather, our intent was to devise a simple approach with a very wide 
range of applicability— and through its simplicity, to encourage its use. (u) 

Technical Description of the Method . To use the method, an analyst must 
first define the application of the technology. By answering what the technology is 
intended to accomplish, the analyst generally has a first listing of parameters to be 
included in the equation. This emphasis on use is important since we believe that 
the state of the art of any technology depends upon how well it fulfills its design 
purposes, (u) 

In most respects an advanced fighter aircraft outperforms a Cessna trainer; 
however, since the role of the trainer is to "fly around the pea patch on a Sunday 
afternoon," the state of the art of the Cessna trainer probably exceeds that of the 
fighter in its particular application. The measures that are important in the 
training mission would be fuel consumption, stability, required control coordina- 
tion, etc. The measures important to the fighter mission would be speed, 

maneuverability, kill potential, range, etc. (u) 

In some instances a single attribute of the technology is all that is required to 
specify its state. This is a pacing parameter; the approach is limiting, but we have 
been able to discover a few cases to which it applies. For example, in 
superconductivity research we might convey a great deal of information about the 
state of that research by simply reporting the highest temperatures at which 
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superconductivity has been achieved. The energy level of particle accelerators 
also provides an example. Similarly, a single measure of the state of telecommu- 
nications might be communication channel capacity expressed in bits/sec. With a 
single indicator as powerful as this, one might draw generalizations about progress 
in telecommunications: (in this instance it has been growing essentially exponen- 
tially over a period of a century or more, the growth as measured by channel 
capacity being an order of magnitude every 17 years or so), (u) 

A bounded measure is one that has an upper and lower bound that is 
determined by theoretical or other constraints. When such bounds are known, the 
current value of the state of the art can be stated in terms of the percentage of 
the limit that has been achieved. Thus, for a class of equipment used in relativistic 
experiments, percentage of speed of light achieved may be a useful state-of-the- 
art measure. Efficiency measures also generally fall into this class since their 
limits are fixed by conservation laws. Resolving power of optical systems is also 
similarly limited, (u) 

It is necessary to choose some means of making each of the parameters in the 
equation nondimensional. This is accomplished by dividing each parameter by some 
reference value of the parameter. Where we are lucky enough to know the 
maximum value that a parameter can ever achieve, the denominator can be that 
maximum value. However, in the usual case, the limit of the parameter (if one 
exists) is unknown and therefore a different approach must be taken. In our past 
work we have experimented with several different conventions. In some instances 
we have utilized the most recent example of the technology as the reference value. 
For example, if we were examining microcomputers, we might take the speed, 
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volume, memory capacity, and other descriptors of the IBM-AT computer and 
utilize these values as the reference values for all the other machines. If the 
weights assigned to the parameter ratios were chosen to add to 1, then the IBM-AT 
itself would receive a state-of-the-art score of unity, and the state of the art of all 
other machines would be measured with respect to the IBM-AT. (u) 

A second approach to nondimensionalizing the parameters is to perform a 
transform for each parameter in which the lowest value achieved by any of the 
technologies in the set is assigned the value of 0, the highest achieved, a value 
of 1. Using this approach, a technology that had all of the lowest parameter values 
would score a zero state of the art, and a technology that had all of the highest 
parameter values would score unity on the state-of-the-art scale, (u) 

A last approach to parameter nondimensionalization is to perform a regres- 
sion on and forecast each parameter individually. This allows the use of either the 
maximum forecasted value or the projected value in any given future year as a 
denominator. All of the above approaches are described further below, (u) 

There are, in general, two methods to determining specifically which 
parameters to include and their associated weights; these are expert judgment and 
statistical methods. In this study, expert judgments were relied on to select 
parameters and their weights. In this approach, experts are asked to provide their 
judgments about the list of factors important to the definition of the frontier of a 
particular technology and to assign weights to each of the factors. The opinions 
are then collated and used as the set of weights for the SOA calculations, (u) 

A number of statistical approaches that aid in identifying unique sets of 
parameters— that is, parameters that do not exhibit collinearity— and for determin- 
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ing appropriate parameter weights also exist. Among these statistical techniques 
are: discriminate analysis,* cluster analysis,** seriation and multidimensional 

scaling,*** covariance analysis and factor analysis. In prior work for the 
NSF,**** The Futures Group has developed a particularly interesting iterative 
approach to estimate the technology used in a state-of-the-art equation. In this 
approach, we begin with an assumption that the state of the art versus time curve 
can be defined by some function such as: (u) 

SOA =(^j[l + tanh A(t-T 0 )]j (2) 

where U is the upper limit of the variable, T 0 is the time of the inflection point, 
and A is a constant that depends on the maximum value of the slope, (u) 

With the idea in mind that the state-of-the-art function follows a known 
curve shape, the complete state-of-the-art equation can be written as follows: (u) 

f * ,anh [A<, - T °>]j ■ c - (?!■)* c 2 c 3 «) 


*Devendra Sahal, "The Generalized Distant Measures of Technology," Tech- 
nological Forecasting and Social Change . Vol. 9 (1976), pp. 289-300. (u) 

♦♦Brian Everitt, Cluster Analysis (London: Heinemann Ed. Books Ltd. 

S.S.R.C., 1979). (u) 

* * * W . S. Torgerson, Theory and Methods of Scaling (New York: John Wiley & 
Sons, Inc., 1958). (u) 

♦♦♦♦Contract No. SRS-8109^0, "Further Research into Technology Output 
Measures," by T. Gordon and K. Edwards, June 1982. (u) 
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where C n is operationally equivalent to the weighting constant of the parameters, 
P n is the value of a parameter for a given computer and P' n is the normalizing 
value for the parameter. Thus, in this formation there are five unknown 
coefficients: A, T 0 , Cj, C2, C3. (u) 

An iterative solution has been used to solve this equation numerically. In this 
approach, values are assumed for the weights, and the state of the art is calculated 
for each of the products or processes under study. State-of-the-art values for each 
of the parameters are fit to a given S-shaped curve. There are a variety of 
formulations one can use in describing an S-shaped curve. Among these, the 
following equation was chosen for use in the SOA methodology: (u) 

LN((V/L)/( 1 -(V/L))) = m *t + b («) 

where V is the value of the variable, L is the upper limit of the variable, m and 
b are constants, and t is time. This equation is estimated for each of the 
technology's parameters, for example, speed, volume and memory size for 
computers. The three parameters in this equation are the intercept, b, the slope, 
m, and the upper limit of the variable, L. The intercept and slope can be estimated 
after a linear transformation is performed on the left-hand side of the equation, 
which contains the values of the parameter under analysis, by fitting a straight line 
through the data sit. The only unknown parameter is the upper limit of the 
variable, L. This upper limit can be derived in a variety of ways. This entire 
procedure, including the derivation of the upper limit, will be explained below in 
the forecasting section of this chapter, (u) 
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The formulation of Equation k was selected for use in the SOA methodology 
because the third parameter, the value the user needs to supply, is a variable that 
has a physical meaning~the upper limit of the parameter. Equation U also is 
symmetrical around the inflection point, making it easier to estimate computa- 
tionally. (u) 

The best-fit values for state of the art are assigned to each of the 
parameters. These values are then related back to their respective S-shaped 
curves, and aggregated to form a total state-of-the-art curve using the previously 
determined values of the weights, (u) 

A curve-fitting module is available with the TFG STATPLAN software 
program. This module can fit a variety of curves to analyze which curve shape 
best describes the given parameter data. The formulations available to test are: (u) 


1. V = m * t * b 

2. L S'(V) s m * t 4 b 

3. L\(V) = m * L\(t) * b 
V s m • LN(t) - b 

3. 1/V r m « t * b 

6. 1/V = m/t * b 

7. Vsm/t*b 

8. L\'((v / 1) / (1 - (V / L))) = 

where: V i value of va-iable 

t — time 

m i coutait 


'1/ 



• k: 


■lk 

■iL 

t 

.... -l/ 

t 

b = constant 

L = upper li r nit of variable (u) 
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Two other interesting formulations of the S-shaped curve examined in past studies 
ares (u) 

SOA = B |t-T 0 J A ( 5 ) 

SOA = B exp |a(T-T 0 )| (6) 

Both equations represent families of monotonically increasing state-of-the- 
art functions with time. In case studies in which rapid increases in the technology 
were occurring (such as computers) little difference was found in the statistical 
estimation of the value of the S-shaped curve when one or the other of the three 
families of functional forms was used. However, for more mature technologies 
that are in fact leveling off in their growth, the choice of the equation form would 
probably be very important, (u) 

The choice of the curve shape to be used on the left side of the equation is 
tantamount to asking the question: "Are technological growth curves bounded or 
unbounded?" There are many technological variables that appear to have growth 
boundaries and within this class, many that appear to be "S" shaped, initially 
growing slowly, reaching some maximum growth rate, and then, passing the 
inflection point, diminishing in growth rate to approach some upper bound 
asymptotically. Several investigators have used this idea in technological forecast- 
ing: Pearl has suggested the use of an S-shaped "logistics curve" that depicts the 
growth in the number of organisms with limited nutrition in a closed environment.* 


*R. C. Lenz, Jr., Technological Forecasting , 2nd ed., Aeronautical Systems 
Division, AFSC, ADS-TDR-62-414, Wright-Patterson AFB, Ohio, June 1962 (AD 408 
085); J. C. Fisher and R. H. Pry, "A Simple Substitution Model of Technological 
Change," Technological Forecasting and Social Change , Vol. 3 (1971), pp. 75-88; 
and A. Wade Blackman, Jr., "The Market Dynamics of Technological Substitutions," 
Technological Forecasting and Social Change , Vol. 6 (1974), pp. 41-63. (u) 


127 



Empirical evidence exists that some parameters follow S-shaped curves. It may 
well be that most technologies grow exponentially at first and then become limited 
either in terms of their growth rate or in terms of their absolute level, but there is 
no well-developed theoretical basis to support these observations. Nevertheless, 
the S-shaped assumption is intuitively satisfying and empirically supported in many 
applications, (u) 

Forecasting State of the Art (u) 

If a technology improves in its capabilities over time, from primitive 
beginnings to maturity, then one might imagine its growth path as following an S- 
shaped curve. If this image of growth is realized, then the asymptote that the S- 
shaped curve approaches defines an upper limit of the technology, that is, an 
"ultimate" level of achievement that the technology can realize. This S-shaped 
picture of technological growth is satisfying: at the beginning, early technologies 
score low on the state-of-the-art scale, and the growth rate is slow. Then, as the 
technology begins to mature, the growth rate accelerates until, at the inflection 
point of the curve, growth rate is at a maximum. Further technological 
developments score higher on the state-of-the-art scale, but their growth rate 
diminishes, until finally, very mature examples of the technology show hardly any 
improvement in state of the art as the upper asymptote of the curve is approached. 
<u) 

This concept of S-shaped growth embodies the implicit assumption that 
technology growth is bounded. A number of authors have pointed out that when 
one technology reaches an upper bound, another based on different operating 
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principles may grow past the limits that stymied the first technology and achieve 
new limits of its own. (u) 

If one has an idea about the shape of the growth curve, then conventional 
curve-fitting techniques can be used to extrapolate the early growth history and 
forecast, for the near term at least, the future growth of the technology. Fitting 
an S-shaped curve to a set of points is a little more challenging than some other 
functions, but one particularly useful way to accomplish this fitting is to perform a 
transform that converts an S-shaped curve into a straight line and then fits the 
straight line to the state-of-the-art points using the method of ordinary least 
squares. The transform is: (u) 

f(t) = In / P/P' \ (7) 

V l-cp/p')/ 

where f(*) is the value of the transformed variable, P is the observed value of the 
parameter and P' is the maximum value of the parameter, (u) 

An example of this approach is shown below. Suppose that our sample 
includes five models of a technology, which have values for the first parameter as 
follows: (u) 


Technology 

Date 

Parameter 1 

1 

1950 

.1 

2 

1953 

.15 

3 

1967 

.25 

it 

1973 

.3 

5 

1980 

.6 
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The transformed values to estimate the state of the art of this parameter are as 
follows: (u) 


Technology (P/P')/(1-(P/P»)) ln((P/P')/(l-(P/P'))) 


1 

2 

3 

4 

5 


.111 

.176 

.333 

.429 

1.500 


-2.197 

-1.734 

-1.099 

-0.847 

0.406 (u) 


Figures 2.1 through 2.3 show these data plotted in three different ways: 
Figure 2.1 presents the data with no transformation plotted on rectilinear 
coordinates. Figure 2.2 shows (P/P')/(1-(P-P')) plotted on semilog coordinates. 
Finally, Figure 2.3 shows In ((P/P*)/ ( 1-(P/P’))) SOA/(l-SOA)) plotted on rectilinear 
coordinates. The solution of this equation thus represents the SOA of the 
individual parameter. This equation is estimated for each of the parameters 
needed to evalute the technology in question, then aggregated using the predeter- 
mined weights to calculate the total SOA index for the technology, (u) 

Once the best fit to the historical points has been determined for each 
parameter, the line can be extrapolated. Of course, such a forecast requires the 
assumptions: (1) that the forces at work in the past will continue to operate in the 
future, and (2) that the SOA of the parameter will continue to follow the S-shaped 
path. The forecast for the given data set is: (u) 

Forecast 

.684 
.819 
.904 

.952 (u) 


Date 

1990 

2000 

2010 

2020 
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Figure 2.1. Rectilinear Plot 
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SOA-1 


In sum, using the straight-line form of the S-shaped curve provides a very 
convenient means of tracking and forecasting the SOA of each parameter j given 
some initial points, (P/P')/(1-(P/P')) is computed for each parameter and plotted on 
semilog coordinates. The best-fit "straight line" is placed through these points 
using an ordinary least squares estimation procedure. The extrapolation of the line 
past the given data points shows a reasonable expectation of how the state of the 
art of the parameter might grow, at least in the near term, if the growth continues 
in an S-shaped form, (u) 

To take a concrete example, suppose we want to forecast the state of the art 
of electric motors. The parameters that describe the state of the art might be 
efficiency and torque, as described by the hypothetical data presented below: 


Date 

Motor Introduced Efficiency Torque 


1 

1930 

.500 

.091 

2 

1935 

.576 

.099 

3 

1937 

.583 

.104 

4 

1942 

.636 

.115 

5 

1947 

.689 

.129 

6 

1950 

.718 

.138 

7 

1953 

.747 

.145 

8 

1960 

.806 

.167 


Suppose further that the weights we want to assign to efficiency and torque are .3 
and .7, respectively. The SOA equation for the technology is: (u) 


SOA = .3 efficiency _ torque 

reference efficiency * reference torque 


(u) 
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The remaining issue is, therefore, the choice of the denominators in this equation. 
In practice, we have used four approaches in determining the denominators, (u) 

1. Specify an upper limit for each variable. 

2. Use the given data to compute the upper limit. 

3. Use the maximum historical value of the parameter as a reference 
(no forecast). 

4. Use the computed value of the variables in 10 years as a reference. 

(u) 

In the third case, the nondimensionalized parameters range between zero and one; 
if the weighting coefficients sum to unity, then the SOA values for the technology 
will also range from zero to one. (u) 

Just how can a maximum possible future value of a parameter be deter- 
mined? There can be theoretical or feasible upper limits, such as a physical upper 
limit of the speed of iight or a limit of 100 percent, if the variable is in percentage 
terms. In these cases, upper limits can easily be specified for each variable. 
Usually, however, the upper limit is not readily apparent, (u) 

Three methods have been developed to determine the upper bound of a 
parameter. The first method is to let the program compute the maximum by 
fitting S-shaped curves with a series of upper limits. If the growth curve of the 
parameter is itself assumed to be S-shaped and the early points are representative, 
the process involves: (u) 

1. Selecting a possible upper limit for test; this test value begins at the 
historical maximum, increasing in steps until a maximum of 10,000 
times the historical maximum is reached. 

2. Using this test value, the historically achieved data points are 
nondimensionalized. 
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( P/P' \ 

3. These are transformed to the In y j-p/p* J space (where P is the 
observed value of the parameter 'and P’^the test value of the 
asymptote). 

4. Absolute errors between the points and the best-fit straight line are 
computed and summed. 

5. A new test value of the asymptote is selected and the process is 
repeated. 

6. Now the summed errors are compared based on some threshold 
criteria; the upper limit that creates the least error is selected as 
the most appropriate, (u) 

In our application, the selection criteria have been: (u) 

- When successively increasing multipliers bring a change in error of 
less than .5 percent, or 

- When successively increasing multipliers result in a minimum value 
in the summed errors, (u) 

This process can be seen graphically in Figure 2.4, where the fit of curves A, 
B and C to the set of datapoints, each with a successively lower limit, is compared. 
The user can then concur with the iteration the program selects or select one of 
the other iterations, (u) 
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This approach works well if there are many data points available for the 
analysis, because then the estimated coefficients are reliable. If there are not 
enough data, however, or if the data are all grouped together at the beginning of 
the S-shaped curve, then the fit of the curve is not as dependable. It is in these 
cases that the analyst must combine expert opinion with the performance of the 
successive iterations to arrive at a reasonable upper limit, (u) 

Another disadvantage of using the approach described above is that if the 
technology of the particular parameter is not very developed, that is, still at the 
beginning of the S-shaped curve, the upper limit generated by the program can be 
quite high. There are two consequences of this. One is that if the upper limits are 
too high, the SOAs are calculated at a very low level. Thus it is difficult to see the 
progress of the technology in the past or in the future because all of the data 
points are bunched together at the lower end of the SOA scale. The other result of 
a high maximum limit is that the weights become less important in the overall SOA 
score because the numbers are so small. For example, if a parameter doubles in 
value from 2 to k, with an upper limit of 1,000, then the doubling in value is not 
very important. If, however, the upper limit had been 8, then the change in value 
becomes very important and that change is dominant in the SOA calculation of the 
parameter. The limit thus becomes more important than the weights, perhaps 
resulting in a different parameter dominating than intended, if it has a smaller 
upper limit, (u) 

In the cases where one parameter is close to its upper limit and the other 
parameters are not, as in the example above, one of two options can be selected to 
correct for the potential imbalance in the SOA figures. The first possibility is to 
use the maximum historical value of the parameter as the reference point. This 
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places all of the parameters on an equal footing because all are in the same range; 
that is, all parameters attain the value of one at some point in the calculation. 
Thus the weights then become the important input to the SOA procedure. The 
disadvantage of this method, however, is that no forecast can be completed for the 
technology, as the SOA has already reached the maximum level for each of the 
parameters, (u) 

Another approach was developed to ensure that parameters are basically in 
the same range and still able to be forecasted. The SOA program goes through the 
same procedure discussed under the second alternative above, that is, using the 
given data to compute successive upper limits. After the best fit is determined by 
the program and the analyst, the parameters are projected over the forecast 
horizon, using their respective maximum limits generated by the most appropriate 
S-shaped curve. The program then uses the parameter value in the tenth year 
projected as the reference point, and recomputes the SOA. Thus the parameters 
are put on the same relative footing, and are forecasted as well. Note that the 
actual forecast numbers for the parameters do not change; the relative positions 
within the SOA calculation change, due to the use of different maximum limits, (u) 

Cross-National Comparisons (u) 

Evaluating U.S. and foreign progress in key technologies has been for some 
time a major methodological challenge for intelligence analysts concerned with 
assessing implications for U.S. security and industrial competitiveness. As our 
previous work in this area demonstrates, the state-of-the-art (SOA) measurement 
method used for this study has shown considerable promise in measuring and 
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tracking technological advances. There seem to be several important advantages 
to the method. First, it gives precision, clarity and replication to the analysis of 
technological progress, gaps and lead time, areas of research that have generally 
lacked each of these qualities. Second, it provides an effective means of blending 
quantitative analysis with qualitative expert judgments on specific U.S. and foreign 
systems and industrial performance. Third, it enables an analyst to track 
systematically foreign state of the art and identify critical component technolo- 
gies. Fourth, it provides a useful tool for effectively explaining technology leads 
and lags and for justifying controls on technology transfer. Finally, it provides a 
tool for analyzing the impact of specific transfers of technology and for helping to 
identify likely future technological advances. This section is an effort to illustrate 
some of these advantages as well as to report on promising areas of refinement and 
possible future development of the method, (u) 

Using state-of-the-art measures permits increased precision in discussing the 
relative technological status of two countries. Consider, for example, the two very 


oinerent situations displayed in Figures 2.5 and 2.6. In Figure 2.5, the technology 
of country A precedes the development of the same technology in country B; 
both state-of-the-art curves are displaced by a constant amount. In Figure 2.6, 
country A initially leads country B. However, A's rate of growth is slower than 


that of country B, so that eventually country B surpasses country A. In 
Figure 2.5, the lead time position of country A over country B is constant; in 
Figure 2.6, the lead time of country A gradually diminishes and then the situation 
reverses, with country B leading country A after the crossover, (u) 
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These curves permit us to identify two different ways of describing the 
technological advantage of one country over another. "Lead time" is defined as the 
time difference between the attainment of a given state of the art by one country, 
as compared with another. It is the horizontal distance between curves A and B, 
in both Figures 2.5 and 2.6. The "state-of-the-art gap" is the vertical distance 
between curves A and B as shown in Figure 2.7. Note that even though lead 
time remains constant in Figure 2.5, the state-of-the-art gap does not. In 
Figure 2.6, both the lead time and state-of-the-art gap vary with time, (u) 

In the analyses that follow, we have assumed that the state-of-the-art curves 
follow a hyperbolic tangent function of the following sort: (u) 

A (3) = ((X- A ( 1 ))/ (A (2) - A (1 ))) (8) 

Y = .5 +.5* (-EXP (-A(3))/(EXP(A(3)) + EXP (-A(3))) *2 + 1) 

In this formulation of hyperbolic tangent function, the first coefficient, A(l), 
represents the midpoint of the curve, that is, the time at which the state-of-the- 
art curve reaches a state-of-the-art level of. 5. This is the inflection point, the 
point at which the curve changes from an increasing growth rate to a decreasing 
growth rate. It also is the point at which growth rate is a maximum. The second 
coefficient, A(2), is the time at which the curve reaches a state-of-the-art value of 
.881; thus, the interval between A(l) and A(2) reflects the steepness of the curve. 
Figure 2.8 illustrates the shape of this function, where A(l) equals 1950 and A(2) 
equal 1960; Figure 2.9 illustrates the function when A(2) is stretched to 1980. (u) 

A computer program was written to explore the properties of S-shaped 
^ curves, particularly those associated with changing lead time and SOA gap. The 

first use of this program explored a constant lead-time situation in which the time 
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between .5 and .881 was 30 years for each of two hypothetical countries. A series 
of runs on U.S. and Soviet aircraft engine SOA was performed to determine the 
effects of varying lead time (that is, the time spacing between the curves). 
Figure 2.10 summarizes the results of these runs and shows how changing lead time 
is reflected in the state-of-the-art gap. (u) 

A second set of runs was completed when the time between SOA = .5 and 
SOA = .881 is only 10 years. The results of this sharper growth case are 
summarized in Figure 2.11. Some very interesting results can be observed from 
these figures, (u) 

At the time the first country reaches a state of the art of .5 (that 
is, 1950), the state-of-the-art gap between the countries is still 
growing. This gap continues to grow until a later time. This 
continuing interval of growth is equal in duration to half of the lead 
time, (u) 

Figure 2.12 shows the maximum state-of-the-art gap that exists as a function of 
lead time for the two cases examined (the sharp-rising case in which ten years 
were assumed to exist between SOA = .5 and .881, and the slow-rising case in which 
30 years were assumed to exist between SOA = .5 and .881). The data in this figure 
are presented in nondimensional form in Figure 2.13 which depicts the relationship 
between the maximum SOA gap, and the ratio of the lead time to the time between 
SOA and .5 and .881. (u) 

For any given curve shape, the maximum SOA gap that will exist 
continues to increase as lead time grows; when the lead time and 
the time between SOA = .5 and .881 are equal, the maximum SOA 
gap will be .$62. (u) 

These relationships are important. Imagine the situation as seen through the 
eyes of country A in Figure 2.5. Suppose this country is interested in maintaining 
its technological lead. Even though it observes the increasing technological 
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Sharp-Rising 


Slow-Rising 


Figure 2.12. Maximum SOA Gap 
versus Lead Time 



Figure 2.13. Relationship Between Maximum SOA Gap, 
Lead Time, and SOA Curve Rise 
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competence of country B, it observes an increasing technological gap until after i, 
has passed an SOA of .5. In fact, country A's position continues to increase until a 
time that is one-half of the lead time after passing SOA = .5. By then, of course, 
the die has already been cast and country B, following the same growth curve, cuts 
the gap ultimately to zero. Country B actually begins to diminish the SOA gap 
before i, reaches its SOA of .5. Country A would say, "How did they do that? 
We kept our lead time constant." True enough, that lead time was kept constant, 
but lead time is most important in terms of state-of-the art gap only near 
maximum SOA growth rate, that is, near SOA = .5. ( u ) 


Now let us examine the situation that occurs when the lead time between 
countries changes, as depicted in Figure 2.6. Figure 2.14 shows the results of a set 
of runs on U.S. and Soviet aircraft engines in which the time between SOA = .5 and 
.881 is different between hypothetical countries. In all instances, the second curve 
has a slower growth, and both curves intersect at SOA = .5. Considering the pre- 
1950 curves, the SOA gap is seen to increase for the slowly growing country until 
just a few years before the faster-growing country "takes off." Then the state-of- 
the-art gap closes rapidly until all the advantage of the slow country is lost. Note 
that the time of reversal of the growth of the gap varies from 7 years from cross- 
over to about 13 years from crossover for the conditions examined in Figure 2.14. 
These runs lead to several conclusions: (u) 


If the growth rates are different, and the higher growth 
the slower, the curves will always cross, (u) 


curve lags 


If the SOA curves intersect at SOA = .5, then the gaps will be 
symmetrical with respect to time before and time after the 
crossover period, (u) 
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Figure 2.14. SOA Gap as a Function of Different Growth Rates 


When two countries have technologies growing at different rates, 
the SOA gap will always hit a maximum and then diminish before 
the crossover period, (u) 

If the SOA curves intersect at SOA = .5, then the country that is 
initially slow, but in the lead, will see a maximum SOA gap at some 
period before crossover. The lead that this country has increases 
slowly at first until the maximum value is reached; then it rapidly 
disappears as the lagging but faster country crosses over and takes 
the lead. The shape of these curves is such that the catch-up phase 
is quite compressed in time and therefore it may be surprising that 
the country that has been in the lead is slower growing, (u) 

So in this situation, the slower-growing but leading country might say, "We 

were tracking our competitor, and our SOA gap was increasing over time, and then 

before we knew it, the gap disappeared." (u) 

For the late but rapidly growing country, immediately after cross- 
over its SOA advantage grows sharply, reaches a maximum and then 
the SOA gap diminishes over time as the slower-growing country 
continues to gain technological competence.* (u) 

In the real world, both lead time and growth rates are likely to vary when 
comparing state of the art of a particular technology between two countries. 
Therefore, to some degree, all of the effects stated above are likely to be 
experienced, (u) 


Sample SOA Run (u) 

The next pages contain a selection of the output from a sample SOA run. 
Table 2.1 shows the tabular results of the total SOA solution for 10-100 kW 
generators, (u) 


♦One reviewer, upon seeing this description of SOA gap resulting from 
differential technological growth rates, commented that it aptly described the 
pattern existing between the United States and Japan in the automobile industry. 
The United States had an SOA lead for a long time. Japan's growth in this industry 
was very rapid, and almost before it was recognized by the United States, the SOA 
gap diminished and in some ways the Japanese took leadership. Now, with steady 
but slow continued technological growth in the United States, Japan's lead is being 
eroded again, (u) 
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10-100 KW AC GENERATORS (u) 
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The SO A paths of 10-100 kW generators for the U.S. and the USSR are 
indicated in Figure 2.15. The relative paths can be traced only until 1982, which is 
the last available U.S. data point. Using the U.S. generator MEP404A in 1969 as a 
reference point, the SOA gap between the U.S. and the USSR is the vertical 
distance between the MEP404A and the Soviet SOA line-a gap of approximately 
.15 in 1969. The lag between the two countries is the difference in time from when 
the U.S. has attained a certain state of the art to when the USSR achieves the 
same level, (u) 
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YEARS IN WHICH 
SPECIFIC MODELS 
WERE INTRODUCED 


Parameter 


65 



Weight 


85 (u 


Upper 

Limit 


SUMMARY of PARAMETERS, 
WEIGHTS, AND UPPER 
LIMITS USED IN SOA 
CALCULATION 


Power t Weight (watts per 
.pound) 

Efficiency Measure (energy 
produced (Kwh) per gallon 
of fuel ) 


.40 


.35 


126.0 

20.0 


| LEGEND 


Power t Volume (kilowatts 
per cubic foot) 

1 U.S. Development 

k USSR Development 


.25 


3.2 (u) 


Caveats: 


ISSUES THAT 
MAY QUALIFY 
THE SOA 
RESULTS 


Several desirable parameters were not included due to lack of 
sufficient data: reliability, running time, noise level, infrared 
and electromagnetic emissions, quality of electric output and 
ruggedness. 


Some dates are approximate. 

Data on recent Soviet developments, including possibly two gas 
turbines, was not available. 


(u 


Figure 2.15 


Forecast 

(. 48 ) 
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